GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → moraieu → query-selector

moraieu / query-selector

Licence: Apache-2.0 license
LONG-TERM SERIES FORECASTING WITH QUERYSELECTOR – EFFICIENT MODEL OF SPARSEATTENTION

Programming Languages

python
139335 projects - #7 most used programming language

Labels

machine-learningdeep-learningtime-seriespytorchtransformerforecastingself-attention

Projects that are alternatives of or similar to query-selector

Flow Forecast
Deep learning PyTorch library for time series forecasting, classification, and anomaly detection (originally for flood forecasting).
Stars: ✭ 368 (+484.13%)
Mutual labels:  time-series, transformer, forecasting
Anticipy
A Python library for time series forecasting
Stars: ✭ 71 (+12.7%)
Mutual labels:  time-series, forecasting
Neural prophet
NeuralProphet - A simple forecasting model based on Neural Networks in PyTorch
Stars: ✭ 1,125 (+1685.71%)
Mutual labels:  time-series, forecasting
Forecasting
Time Series Forecasting Best Practices & Examples
Stars: ✭ 2,123 (+3269.84%)
Mutual labels:  time-series, forecasting
Informer2020
The GitHub repository for the paper "Informer" accepted by AAAI 2021.
Stars: ✭ 771 (+1123.81%)
Mutual labels:  time-series, forecasting
Pmdarima
A statistical library designed to fill the void in Python's time series analysis capabilities, including the equivalent of R's auto.arima function.
Stars: ✭ 838 (+1230.16%)
Mutual labels:  time-series, forecasting
Statespacemodels.jl
StateSpaceModels.jl is a Julia package for time-series analysis using state-space models.
Stars: ✭ 116 (+84.13%)
Mutual labels:  time-series, forecasting
Web Traffic Forecasting
Kaggle | Web Traffic Forecasting 📈
Stars: ✭ 596 (+846.03%)
Mutual labels:  time-series, forecasting
Pyfts
An open source library for Fuzzy Time Series in Python
Stars: ✭ 154 (+144.44%)
Mutual labels:  time-series, forecasting
Java Timeseries
Time series analysis in Java
Stars: ✭ 155 (+146.03%)
Mutual labels:  time-series, forecasting
seq2seq-pytorch
Sequence to Sequence Models in PyTorch
Stars: ✭ 41 (-34.92%)
Mutual labels:  transformer, self-attention
Darts
A python library for easy manipulation and forecasting of time series.
Stars: ✭ 760 (+1106.35%)
Mutual labels:  time-series, forecasting
Getting Things Done With Pytorch
Jupyter Notebook tutorials on solving real-world problems with Machine Learning & Deep Learning using PyTorch. Topics: Face detection with Detectron 2, Time Series anomaly detection with LSTM Autoencoders, Object Detection with YOLO v5, Build your first Neural Network, Time Series forecasting for Coronavirus daily cases, Sentiment Analysis with BERT.
Stars: ✭ 738 (+1071.43%)
Mutual labels:  time-series, transformer
Msgarch
MSGARCH R Package
Stars: ✭ 51 (-19.05%)
Mutual labels:  time-series, forecasting
Arch
ARCH models in Python
Stars: ✭ 660 (+947.62%)
Mutual labels:  time-series, forecasting
Forecastml
An R package with Python support for multi-step-ahead forecasting with machine learning and deep learning algorithms
Stars: ✭ 101 (+60.32%)
Mutual labels:  time-series, forecasting
Modeltime
Modeltime unlocks time series forecast models and machine learning in one framework
Stars: ✭ 189 (+200%)
Mutual labels:  time-series, forecasting
Sktime
A unified framework for machine learning with time series
Stars: ✭ 4,741 (+7425.4%)
Mutual labels:  time-series, forecasting
Tsai
Time series Timeseries Deep Learning Pytorch fastai - State-of-the-art Deep Learning with Time Series and Sequences in Pytorch / fastai
Stars: ✭ 407 (+546.03%)
Mutual labels:  time-series, transformer
Gluon Ts
Probabilistic time series modeling in Python
Stars: ✭ 2,373 (+3666.67%)
Mutual labels:  time-series, forecasting
View All Similar Projects ➔

Query Selector

Here you can find code and data loaders for the paper https://arxiv.org/pdf/2107.08687v1.pdf . Query Selector is a novel approach to sparse attention Transformer algorithm that is especially suitable for long term time series forecasting

Depencency

Python            3.7.9
deepspeed         0.4.0
numpy             1.20.3
pandas            1.2.4
scipy             1.6.3
tensorboardX      1.8
torch             1.7.1
torchaudio        0.7.2
torchvision       0.8.2
tqdm              4.61.0

Results on ETT dataset

Univariate

Data Prediction len Informer MSE Informer MAE Trans former MSE Trans former MAE Query Selector MSE Query Selector MAE MSE ratio
ETTh1 24 0.0980 0.2470 0.0548 0.1830 0.0436 0.1616 0.445
ETTh1 48 0.1580 0.3190 0.0740 0.2144 0.0721 0.2118 0.456
ETTh1 168 0.1830 0.3460 0.1049 0.2539 0.0935 0.2371 0.511
ETTh1 336 0.2220 0.3870 0.1541 0.3201 0.1267 0.2844 0.571
ETTh1 720 0.2690 0.4350 0.2501 0.4213 0.2136 0.3730 0.794
ETTh2 24 0.0930 0.2400 0.0999 0.2479 0.0843 0.2239 0.906
ETTh2 48 0.1550 0.3140 0.1218 0.2763 0.1117 0.2622 0.721
ETTh2 168 0.2320 0.3890 0.1974 0.3547 0.1753 0.3322 0.756
ETTh2 336 0.2630 0.4170 0.2191 0.3805 0.2088 0.3710 0.794
ETTh2 720 0.2770 0.4310 0.2853 0.4340 0.2585 0.4130 0.933
ETTm1 24 0.0300 0.1370 0.0143 0.0894 0.0139 0.0870 0.463
ETTm1 48 0.0690 0.2030 0.0328 0.1388 0.0342 0.1408 0.475
ETTm1 96 0.1940 0.2030 0.0695 0.2085 0.0702 0.2100 0.358
ETTm1 288 0.4010 0.5540 0.1316 0.2948 0.1548 0.3240 0.328
ETTm1 672 0.5120 0.6440 0.1728 0.3437 0.1735 0.3427 0.338

Multivariate

Data Prediction len Informer MSE Informer MAE Trans former MSE Trans former MAE Query Selector MSE Query Selector MAE MSE ratio
ETTh1 24 0.5770 0.5490 0.4496 0.4788 0.4226 0.4627 0.732
ETTh1 48 0.6850 0.6250 0.4668 0.4968 0.4581 0.4878 0.669
ETTh1 168 0.9310 0.7520 0.7146 0.6325 0.6835 0.6088 0.734
ETTh1 336 1.1280 0.8730 0.8321 0.7041 0.8503 0.7039 0.738
ETTh1 720 1.2150 0.8960 1.1080 0.8399 1.1150 0.8428 0.912
ETTh2 24 0.7200 0.6650 0.4237 0.5013 0.4124 0.4864 0.573
ETTh2 48 1.4570 1.0010 1.5220 0.9488 1.4074 0.9317 0.966
ETTh2 168 3.4890 1.5150 1.6225 0.9726 1.7385 1.0125 0.465
ETTh2 336 2.7230 1.3400 2.6617 1.2189 2.3168 1.1859 0.851
ETTh2 720 3.4670 1.4730 3.1805 1.3668 3.0664 1.3084 0.884
ETTm1 24 0.3230 0.3690 0.3150 0.3886 0.3351 0.3875 0.975
ETTm1 48 0.4940 0.5030 0.4454 0.4620 0.4726 0.4702 0.902
ETTm1 96 0.6780 0.6140 0.4641 0.4823 0.4543 0.4831 0.670
ETTm1 288 1.0560 0.7860 0.6814 0.6312 0.6185 0.5991 0.586
ETTm1 672 1.1920 0.9260 1.1365 0.8572 1.1273 0.8412 0.946

State Of Art

PWC

PWC

PWC

PWC

PWC

PWC

PWC

PWC

PWC

PWC

Citation

@misc{klimek2021longterm,
      title={Long-term series forecasting with Query Selector -- efficient model of sparse attention}, 
      author={Jacek Klimek and Jakub Klimek and Witold Kraskiewicz and Mateusz Topolewski},
      year={2021},
      eprint={2107.08687},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Contact

If you have any questions please contact us by email - [email protected]

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].