Update readme
Browse files
README.md
CHANGED
@@ -1,136 +1,50 @@
|
|
1 |
---
|
|
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
|
5 |
-
# Moirai-1.0-R-Small
|
|
|
|
|
6 |
|
7 |
Moirai, the Masked Encoder-based Universal Time Series Forecasting Transformer is a Large Time Series Model pre-trained on [LOTSA data](https://huggingface.co/datasets/Salesforce/lotsa_data).
|
8 |
-
For more details on the Moirai architecture, training, and results, please refer to the [paper](https://arxiv.org/abs/2402.02592).
|
9 |
|
10 |
<p align="center">
|
11 |
-
<img src="figures/architecture.png" width="100%">
|
12 |
<br />
|
13 |
<span>
|
14 |
Fig. 1: Overall architecture of Moirai. Visualized is a 3-variate time series, where variates 0 and 1 are target variables (i.e. to be forecasted, and variate 2 is a dynamic covariate (values in forecast horizon known). Based on a patch size of 64, each variate is patchified into 3 tokens. The patch embeddings along with sequence and variate id are fed into the Transformer. The shaded patches represent the forecast horizon to be forecasted, whose corresponding output representations are mapped into the mixture distribution parameters.
|
15 |
</span>
|
16 |
</p>
|
17 |
|
18 |
-
##
|
19 |
-
|
20 |
-
To perform inference with Moirai, install the uni2ts library from our [GitHub repo](https://github.com/SalesforceAIResearch/uni2ts).
|
21 |
-
|
22 |
-
1. Clone repository:
|
23 |
-
```shell
|
24 |
-
git clone https://github.com/SalesforceAIResearch/uni2ts.git
|
25 |
-
cd uni2ts
|
26 |
-
```
|
27 |
-
|
28 |
-
2) Create virtual environment:
|
29 |
-
```shell
|
30 |
-
virtualenv venv
|
31 |
-
. venv/bin/activate
|
32 |
-
```
|
33 |
-
|
34 |
-
3) Build from source:
|
35 |
-
```shell
|
36 |
-
pip install -e '.[notebook]'
|
37 |
-
```
|
38 |
-
|
39 |
-
4) Create a `.env` file:
|
40 |
-
```shell
|
41 |
-
touch .env
|
42 |
-
```
|
43 |
-
|
44 |
-
A simple example to get started:
|
45 |
|
46 |
```python
|
47 |
-
import
|
48 |
-
|
49 |
-
from gluonts.dataset.pandas import PandasDataset
|
50 |
-
from gluonts.dataset.split import split
|
51 |
-
from huggingface_hub import hf_hub_download
|
52 |
-
|
53 |
-
from uni2ts.eval_util.plot import plot_single
|
54 |
-
from uni2ts.model.moirai import MoiraiForecast
|
55 |
|
|
|
56 |
|
57 |
-
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
BSZ = 32 # batch size: any positive integer
|
62 |
-
TEST = 100 # test set length: any positive integer
|
63 |
|
64 |
-
|
65 |
-
|
66 |
-
"https://gist.githubusercontent.com/rsnirwan/c8c8654a98350fadd229b00167174ec4"
|
67 |
-
"/raw/a42101c7786d4bc7695228a0f2c8cea41340e18f/ts_wide.csv"
|
68 |
-
)
|
69 |
-
df = pd.read_csv(url, index_col=0, parse_dates=True)
|
70 |
-
|
71 |
-
# Convert into GluonTS dataset
|
72 |
-
ds = PandasDataset(dict(df))
|
73 |
-
|
74 |
-
# Split into train/test set
|
75 |
-
train, test_template = split(
|
76 |
-
ds, offset=-TEST
|
77 |
-
) # assign last TEST time steps as test set
|
78 |
-
|
79 |
-
# Construct rolling window evaluation
|
80 |
-
test_data = test_template.generate_instances(
|
81 |
-
prediction_length=PDT, # number of time steps for each prediction
|
82 |
-
windows=TEST // PDT, # number of windows in rolling window evaluation
|
83 |
-
distance=PDT, # number of time steps between each window - distance=PDT for non-overlapping windows
|
84 |
-
)
|
85 |
-
|
86 |
-
# Prepare pre-trained model by downloading model weights from huggingface hub
|
87 |
-
model = MoiraiForecast.load_from_checkpoint(
|
88 |
-
checkpoint_path=hf_hub_download(
|
89 |
-
repo_id=f"Salesforce/moirai-R-{SIZE}", filename="model.ckpt"
|
90 |
-
),
|
91 |
-
prediction_length=PDT,
|
92 |
-
context_length=CTX,
|
93 |
-
patch_size=PSZ,
|
94 |
-
num_samples=100,
|
95 |
-
target_dim=1,
|
96 |
-
feat_dynamic_real_dim=ds.num_feat_dynamic_real,
|
97 |
-
past_feat_dynamic_real_dim=ds.num_past_feat_dynamic_real,
|
98 |
-
map_location="cuda:0" if torch.cuda.is_available() else "cpu",
|
99 |
-
)
|
100 |
-
|
101 |
-
predictor = model.create_predictor(batch_size=BSZ)
|
102 |
-
forecasts = predictor.predict(test_data.input)
|
103 |
-
|
104 |
-
input_it = iter(test_data.input)
|
105 |
-
label_it = iter(test_data.label)
|
106 |
-
forecast_it = iter(forecasts)
|
107 |
-
|
108 |
-
inp = next(input_it)
|
109 |
-
label = next(label_it)
|
110 |
-
forecast = next(forecast_it)
|
111 |
-
|
112 |
-
plot_single(
|
113 |
-
inp,
|
114 |
-
label,
|
115 |
-
forecast,
|
116 |
-
context_length=200,
|
117 |
-
name="pred",
|
118 |
-
show_label=True,
|
119 |
-
)
|
120 |
```
|
121 |
|
122 |
-
##
|
123 |
-
|
124 |
| # Model | # Parameters |
|
125 |
| :---: | :---: |
|
126 |
-
| [Moirai-1.0-R-Small](https://huggingface.co/
|
127 |
-
| [Moirai-1.0-R-Base](https://huggingface.co/
|
128 |
-
| [Moirai-1.0-R-Large](https://huggingface.co/
|
129 |
|
130 |
-
|
131 |
-
|
132 |
-
If you're using Uni2TS in your research or applications, please cite it using this BibTeX:
|
133 |
|
|
|
134 |
```markdown
|
135 |
@article{woo2024unified,
|
136 |
title={Unified Training of Universal Time Series Forecasting Transformers},
|
|
|
1 |
---
|
2 |
+
title: MOIRAI readme
|
3 |
license: apache-2.0
|
4 |
---
|
5 |
|
6 |
+
# Moirai-1.0-R-Small snapshot of [commit](https://huggingface.co/Salesforce/moirai-1.0-R-large/commit/001acc2aca5fb1d5023e2664bbe45471d8baf21a).
|
7 |
+
|
8 |
+
A snapshot of the last Moirai-1.0-R-Small weights released under permissive license, hosted by sktime. For use with the sktime Moirai interface to ensure permissive license throughout the stack
|
9 |
|
10 |
Moirai, the Masked Encoder-based Universal Time Series Forecasting Transformer is a Large Time Series Model pre-trained on [LOTSA data](https://huggingface.co/datasets/Salesforce/lotsa_data).
|
11 |
+
For more details on the Moirai architecture, training, and results, please refer to the [paper](https://arxiv.org/abs/2402.02592).
|
12 |
|
13 |
<p align="center">
|
14 |
+
<img src="https://huggingface.co/Salesforce/moirai-1.0-R-large/resolve/main/figures/architecture.png", width="100%">
|
15 |
<br />
|
16 |
<span>
|
17 |
Fig. 1: Overall architecture of Moirai. Visualized is a 3-variate time series, where variates 0 and 1 are target variables (i.e. to be forecasted, and variate 2 is a dynamic covariate (values in forecast horizon known). Based on a patch size of 64, each variate is patchified into 3 tokens. The patch embeddings along with sequence and variate id are fed into the Transformer. The shaded patches represent the forecast horizon to be forecasted, whose corresponding output representations are mapped into the mixture distribution parameters.
|
18 |
</span>
|
19 |
</p>
|
20 |
|
21 |
+
## Example
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
|
23 |
```python
|
24 |
+
from sktime.datasets import load_tecator
|
25 |
+
from sktime.forecasting.moirai_forecaster import MOIRAIForecaster
|
|
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
+
y, _ = load_tecator(return_X_y=True, return_type="pd-multiindex")
|
28 |
|
29 |
+
moirai_forecaster = MOIRAIForecaster(
|
30 |
+
checkpoint_path=f"sktime/moirai-1.0-R-small",
|
31 |
+
broadcasting=False
|
32 |
+
)
|
|
|
|
|
33 |
|
34 |
+
moirai_forecaster.fit(y)
|
35 |
+
forecast = moirai_forecaster.predict(fh=range(1, 16))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
36 |
```
|
37 |
|
38 |
+
## Moirai weights hosted under sktime
|
|
|
39 |
| # Model | # Parameters |
|
40 |
| :---: | :---: |
|
41 |
+
| [Moirai-1.0-R-Small](https://huggingface.co/sktime/moirai-1.0-R-small) | 14m |
|
42 |
+
| [Moirai-1.0-R-Base](https://huggingface.co/sktime/moirai-1.0-R-base) | 91m |
|
43 |
+
| [Moirai-1.0-R-Large](https://huggingface.co/sktime/moirai-1.0-R-large) | 311m |
|
44 |
|
45 |
+
Original Weights present at [Salesforce collection](https://huggingface.co/collections/Salesforce/moirai-10-r-models-65c8d3a94c51428c300e0742).
|
|
|
|
|
46 |
|
47 |
+
## Citation
|
48 |
```markdown
|
49 |
@article{woo2024unified,
|
50 |
title={Unified Training of Universal Time Series Forecasting Transformers},
|