Pranavv commited on
Commit
ef3e081
·
1 Parent(s): 4085728

Update readme

Browse files
Files changed (1) hide show
  1. README.md +22 -108
README.md CHANGED
@@ -1,136 +1,50 @@
1
  ---
 
2
  license: apache-2.0
3
  ---
4
 
5
- # Moirai-1.0-R-Small
 
 
6
 
7
  Moirai, the Masked Encoder-based Universal Time Series Forecasting Transformer is a Large Time Series Model pre-trained on [LOTSA data](https://huggingface.co/datasets/Salesforce/lotsa_data).
8
- For more details on the Moirai architecture, training, and results, please refer to the [paper](https://arxiv.org/abs/2402.02592).
9
 
10
  <p align="center">
11
- <img src="figures/architecture.png" width="100%">
12
  <br />
13
  <span>
14
  Fig. 1: Overall architecture of Moirai. Visualized is a 3-variate time series, where variates 0 and 1 are target variables (i.e. to be forecasted, and variate 2 is a dynamic covariate (values in forecast horizon known). Based on a patch size of 64, each variate is patchified into 3 tokens. The patch embeddings along with sequence and variate id are fed into the Transformer. The shaded patches represent the forecast horizon to be forecasted, whose corresponding output representations are mapped into the mixture distribution parameters.
15
  </span>
16
  </p>
17
 
18
- ## Usage
19
-
20
- To perform inference with Moirai, install the uni2ts library from our [GitHub repo](https://github.com/SalesforceAIResearch/uni2ts).
21
-
22
- 1. Clone repository:
23
- ```shell
24
- git clone https://github.com/SalesforceAIResearch/uni2ts.git
25
- cd uni2ts
26
- ```
27
-
28
- 2) Create virtual environment:
29
- ```shell
30
- virtualenv venv
31
- . venv/bin/activate
32
- ```
33
-
34
- 3) Build from source:
35
- ```shell
36
- pip install -e '.[notebook]'
37
- ```
38
-
39
- 4) Create a `.env` file:
40
- ```shell
41
- touch .env
42
- ```
43
-
44
- A simple example to get started:
45
 
46
  ```python
47
- import torch
48
- import pandas as pd
49
- from gluonts.dataset.pandas import PandasDataset
50
- from gluonts.dataset.split import split
51
- from huggingface_hub import hf_hub_download
52
-
53
- from uni2ts.eval_util.plot import plot_single
54
- from uni2ts.model.moirai import MoiraiForecast
55
 
 
56
 
57
- SIZE = "small" # model size: choose from {'small', 'base', 'large'}
58
- PDT = 20 # prediction length: any positive integer
59
- CTX = 200 # context length: any positive integer
60
- PSZ = "auto" # patch size: choose from {"auto", 8, 16, 32, 64, 128}
61
- BSZ = 32 # batch size: any positive integer
62
- TEST = 100 # test set length: any positive integer
63
 
64
- # Read data into pandas DataFrame
65
- url = (
66
- "https://gist.githubusercontent.com/rsnirwan/c8c8654a98350fadd229b00167174ec4"
67
- "/raw/a42101c7786d4bc7695228a0f2c8cea41340e18f/ts_wide.csv"
68
- )
69
- df = pd.read_csv(url, index_col=0, parse_dates=True)
70
-
71
- # Convert into GluonTS dataset
72
- ds = PandasDataset(dict(df))
73
-
74
- # Split into train/test set
75
- train, test_template = split(
76
- ds, offset=-TEST
77
- ) # assign last TEST time steps as test set
78
-
79
- # Construct rolling window evaluation
80
- test_data = test_template.generate_instances(
81
- prediction_length=PDT, # number of time steps for each prediction
82
- windows=TEST // PDT, # number of windows in rolling window evaluation
83
- distance=PDT, # number of time steps between each window - distance=PDT for non-overlapping windows
84
- )
85
-
86
- # Prepare pre-trained model by downloading model weights from huggingface hub
87
- model = MoiraiForecast.load_from_checkpoint(
88
- checkpoint_path=hf_hub_download(
89
- repo_id=f"Salesforce/moirai-R-{SIZE}", filename="model.ckpt"
90
- ),
91
- prediction_length=PDT,
92
- context_length=CTX,
93
- patch_size=PSZ,
94
- num_samples=100,
95
- target_dim=1,
96
- feat_dynamic_real_dim=ds.num_feat_dynamic_real,
97
- past_feat_dynamic_real_dim=ds.num_past_feat_dynamic_real,
98
- map_location="cuda:0" if torch.cuda.is_available() else "cpu",
99
- )
100
-
101
- predictor = model.create_predictor(batch_size=BSZ)
102
- forecasts = predictor.predict(test_data.input)
103
-
104
- input_it = iter(test_data.input)
105
- label_it = iter(test_data.label)
106
- forecast_it = iter(forecasts)
107
-
108
- inp = next(input_it)
109
- label = next(label_it)
110
- forecast = next(forecast_it)
111
-
112
- plot_single(
113
- inp,
114
- label,
115
- forecast,
116
- context_length=200,
117
- name="pred",
118
- show_label=True,
119
- )
120
  ```
121
 
122
- ## The Moirai Family
123
-
124
  | # Model | # Parameters |
125
  | :---: | :---: |
126
- | [Moirai-1.0-R-Small](https://huggingface.co/Salesforce/moirai-1.0-R-small) | 14m |
127
- | [Moirai-1.0-R-Base](https://huggingface.co/Salesforce/moirai-1.0-R-base) | 91m |
128
- | [Moirai-1.0-R-Large](https://huggingface.co/Salesforce/moirai-1.0-R-large) | 311m |
129
 
130
- ## Citation
131
-
132
- If you're using Uni2TS in your research or applications, please cite it using this BibTeX:
133
 
 
134
  ```markdown
135
  @article{woo2024unified,
136
  title={Unified Training of Universal Time Series Forecasting Transformers},
 
1
  ---
2
+ title: MOIRAI readme
3
  license: apache-2.0
4
  ---
5
 
6
+ # Moirai-1.0-R-Small snapshot of [commit](https://huggingface.co/Salesforce/moirai-1.0-R-large/commit/001acc2aca5fb1d5023e2664bbe45471d8baf21a).
7
+
8
+ A snapshot of the last Moirai-1.0-R-Small weights released under permissive license, hosted by sktime. For use with the sktime Moirai interface to ensure permissive license throughout the stack
9
 
10
  Moirai, the Masked Encoder-based Universal Time Series Forecasting Transformer is a Large Time Series Model pre-trained on [LOTSA data](https://huggingface.co/datasets/Salesforce/lotsa_data).
11
+ For more details on the Moirai architecture, training, and results, please refer to the [paper](https://arxiv.org/abs/2402.02592).
12
 
13
  <p align="center">
14
+ <img src="https://huggingface.co/Salesforce/moirai-1.0-R-large/resolve/main/figures/architecture.png", width="100%">
15
  <br />
16
  <span>
17
  Fig. 1: Overall architecture of Moirai. Visualized is a 3-variate time series, where variates 0 and 1 are target variables (i.e. to be forecasted, and variate 2 is a dynamic covariate (values in forecast horizon known). Based on a patch size of 64, each variate is patchified into 3 tokens. The patch embeddings along with sequence and variate id are fed into the Transformer. The shaded patches represent the forecast horizon to be forecasted, whose corresponding output representations are mapped into the mixture distribution parameters.
18
  </span>
19
  </p>
20
 
21
+ ## Example
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
  ```python
24
+ from sktime.datasets import load_tecator
25
+ from sktime.forecasting.moirai_forecaster import MOIRAIForecaster
 
 
 
 
 
 
26
 
27
+ y, _ = load_tecator(return_X_y=True, return_type="pd-multiindex")
28
 
29
+ moirai_forecaster = MOIRAIForecaster(
30
+ checkpoint_path=f"sktime/moirai-1.0-R-small",
31
+ broadcasting=False
32
+ )
 
 
33
 
34
+ moirai_forecaster.fit(y)
35
+ forecast = moirai_forecaster.predict(fh=range(1, 16))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  ```
37
 
38
+ ## Moirai weights hosted under sktime
 
39
  | # Model | # Parameters |
40
  | :---: | :---: |
41
+ | [Moirai-1.0-R-Small](https://huggingface.co/sktime/moirai-1.0-R-small) | 14m |
42
+ | [Moirai-1.0-R-Base](https://huggingface.co/sktime/moirai-1.0-R-base) | 91m |
43
+ | [Moirai-1.0-R-Large](https://huggingface.co/sktime/moirai-1.0-R-large) | 311m |
44
 
45
+ Original Weights present at [Salesforce collection](https://huggingface.co/collections/Salesforce/moirai-10-r-models-65c8d3a94c51428c300e0742).
 
 
46
 
47
+ ## Citation
48
  ```markdown
49
  @article{woo2024unified,
50
  title={Unified Training of Universal Time Series Forecasting Transformers},