Label Correlation Biases Direct Time Series Forecast
Time series;Long-term Forecast
Yes
No ethics review needed.
1. In the experiments, how to determine the hyperparameters of FreDF and baselines?
2. According to the ablation study, the frequency term seems to be almost all that's needed. Is it correct?
3. During my attempt to reproduce the experiments, I encountered an error (`AttributeError: 'Exp_Short_Term_Forecast' object has no attribute 'seasonal_patterns'`) when running the short-term forecasting experiments. Is this error expected, or does it indicate a potential issue in the codebase?

Overall I think this work is interesting and promising. I will consider raising my score if my questions could receive positive responses, especially my major concerns (W1, Q1, Q3).
1. The authors identify and address the bias introduced by label correlation in time-series modeling, which is a novel issue for me and holds substantial potential generality across different scenarios.
2. The method is sound, straightforward and shows very promising results. The theoretical results are relatively persuasive, demonstrating the bias caused by label correlation, and subsequently FreDF's elimination of label correlation and thereby bias.
3. The experiment is comprehensive. Extensive set of experiments showed that: the approach contributes to the state-of-the-art, different components of the loss contribute to performance, the approach is robust to hyperparameter values.
This work introduces a new loss term for multi-step time-series forecasting that penalizes errors within a decorrelated representation space of the ground truth labels. In its current formulation, this decorrelated space is defined as the frequency representation of both labels and forecasts. Experimental results demonstrate that the proposed approach significantly enhances forecasting accuracy across various datasets and base models.
1. It seems that this work aims to penalize errors in a space that decorrelates labels. Experiments mainly achieve de-correlation via FFT or FFT2. It will be beneficial to explore the efficacy of other transformations beyond the Fourier transform?
2. While the introduction discusses various forecasting models, it could benefit from a stronger focus on established forecasting paradigms (e.g., direct forecasting, iterative forecasting). The inclusion of forecast models (iTransformer, Linear) may detract from highlighting the contribution and role of FreDF that seems to be orthogonal to forecast models.
3. The source code should be refined. The current implementation comprises numerous scripts, and it is somewhat unclear how each script relates to the experiments discussed in the manuscript. Besides, the current environment setup appears to depend on unexpected repositories like `torchmetrics` and `patool`. Including a `Dockerfile` or `conda` environment file would help a lot.
Yes
No ethics review needed.
1. Computational complexity of frequency domain conversion: Will the frequency domain conversion mentioned in the paper significantly increase computational complexity? How efficient is the FreDF method when dealing with large-scale datasets?
2. Generalization ability: How does the FreDF method generalize across different domains and datasets? Has it been tested and validated in more practical application scenarios?
3. Applicability of frequency domain features: How effective is the FreDF method for data with unclear or difficult to extract frequency domain features? Are there any relevant experimental results or analysis?
4. Model interpretability: Does the FreDF method have interpretability in the frequency domain? How do users understand and interpret these predicted results?
1. Solving the problem of autocorrelation in label sequences: The FreDF method proposed in this paper effectively bypasses the autocorrelation problem in label sequences by predicting in the frequency domain. This is a common but not fully addressed problem in existing direct prediction models.
2. Compatible with multiple prediction models: The FreDF method is not only suitable for existing state-of-the-art methods such as iTransformer, but can also be compatible with multiple prediction models. This compatibility makes it widely applicable in different prediction tasks.
3. Significant improvement in predictive performance: Experimental results show that the FreDF method is significantly superior to existing methods in multi-step prediction tasks. This indicates that the method has high accuracy and reliability in processing complex time series data.
4. Innovative applications of frequency domain analysis: By introducing frequency domain analysis into time series prediction, the FreDF method provides a new perspective to address autocorrelation issues in time series. This innovative application not only improves predictive performance, but also provides new directions for future research.
Considering that in current time series prediction, DF only reuses general multi task learning methods without considering the dependency relationships  between labels. Especially, even if there is no correlation between labels, the above loss function can be used to train a multitasking model. When training the model, DF attempts to minimize the error between the predicted label sequence and the true label sequence; This assumes that the label sequence is conditionally independent between different time steps, thereby ignoring the correlation between each time step within the label sequence. So the author uses fast Fourier transform in the frequency domain to transform the data from a temporal perspective to a frequency perspective, in order to suppress autocorrelation. And the theoretical derivation of using frequency domain conversion to suppress time step correlation is given. A loss function combining time domain and frequency domain is designed, and the effectiveness of the design method is demonstrated by experimental results.
1. Complexity of frequency domain conversion: Although frequency domain analysis can bypass the autocorrelation problem in label sequences, frequency domain conversion itself may increase computational complexity and time cost. This may become a bottleneck when dealing with large-scale datasets.
2. Model generalization ability: Although FreDF performed well in experiments, its generalization ability in different fields and datasets still needs further validation. Especially in practical applications, the model may need to be adjusted and optimized for specific tasks.
3. Dependence on frequency domain features: The FreDF method relies on the extraction and utilization of frequency domain features, which may limit its applicability in certain situations. For example, for data with unclear or difficult to extract frequency domain features, the performance of FreDF may not be as expected.
Yes
No ethics review needed.
Are there any settings where defining the errors by transferring the predictions to the frequency domain is detrimental to the prediction accuracy?
Originality: The idea of augmenting an objective function with a loss in alternate bases that is more robust to value dependences in prediction sequences, in this case Fourier bases, is novel and interesting.

Significance: The extension applies to a subclass of direct forecasting models that predict components of future sequences independently, ignoring dependences that may exist among them.  This makes the method potentially applicable to a variety of SOTA time series models. 

Quality: The authors attempt to analyze the problem of using simple additive error functions for predicting sequences and its fixes via errors in the FFT transformed space both theoretically and experimentally. Extensive experimentation across multiple datasets and tasks show the merits of the proposed approach compared to SOTA baselines.

Other strengths: Exploration of properties and extension of the proposed framework and its benefits, such as, different prediction length, ablation models,  possible new transformations and errors defined on these transformations. 

Code: Authors provide the code for reproducing the experiments.
The paper presents a new Frequency-enhanced Direct Forecast method for time series predictions that aims to improve the predictions generated by direct forecast (DF) models by training the models using a combination of standard MSE errors and errors defined in the FFT transformed space. The new approach is motivated by the fact that the MSE errors for direct forecast (DF) models are biased and do not properly account for correlations in predicted sequences, while errors on the FFT on the predicted sequences may be more robust to such dependencies.  The experiments on multiple datasets and multiple baseline DF models demonstrate the improved performance of the new method over baselines.
Limited analysis of a bias in Theorem 1: Theorem 1 considers only univariate sequences, cross-correlation terms are not accounted for in the Bias formula. The sequence has dimensionality D.  

Notational inconsistency and formula errors in the paper: 
- The use of 'L' is overloaded, it denotes both the length and inputs of the input sequence. 
- Equation 1 has a mistake. Y and Y_hat should be compared on the same indexes. 
- Definition 3.2, uses 'j' instead of “i”  

The main results in the paper do not report performance over distinct random seeds. Given this is an experimental study, reporting the results on multiple distinct random seed could help to understand its sensitivity.

The short-term forecasting task results in the appendix report only qualitative time spans. It would be good to specify the results in terms of forecasting lengths, similar to the Table 1 for long-term forecasting.
Yes
No ethics review needed.
See Weaknesses.
The presentation of the paper is clear and the idea is straightforward.
This paper proposes to measure the distance between time series prediction outputs and target signals in the frequency domain. 
As claimed, this allows to capture label autocorrelation in time series prediction tasks. Speciafically, in practice, distances in both time and frequency domains are utilized for training in a combined manner.
The proposed approach lacks novelty. Measuring time series structure (e.g., autocorrelation) in the frequency domain has been well-explored. Several prior works have investigated capturing time series characteristics or enhancing robustness in loss functions, including:
  1. Incorporating Fourier transformation in loss functions [1,2].
  1. Employing DTW-based loss to keep shape information of time series [3,4].
   1. Utilizing multiresolution trends during training [5,6].

These approaches are all relevant to this study. However, this paper does not adequately investigate, discuss, or compare these related methods.

[1] Henning Lange, et al. From Fourier to Koopman: Spectral Methods for Long-term Time Series Prediction. JMLR 2021.

[2] Xinyu Yuan and Yan Qiao. Diffusion-TS: Interpretable Diffusion for General Time Series Generation. In ICLR 2024

[3] Vincent Le Guen and Nicolas Thome. Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models. In NeurIPS 2019.

[4] Vincent Le Guen and Nicolas Thome. Probabilistic time series forecasting with shape and temporal diversity. In NeurIPS 2020.

[5] Shiyu Wang, et al. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. In ICLR 2024.

[6] Amin Shabani, et al. Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting. In ICLR 2023.
Learning to forecast in the frequency domain significantly enhances forecasting performance.
@inproceedings{
anonymous2024label,
title={Label Correlation Biases Direct Time Series Forecast},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=4A9IdSa1ul},
note={under review}
}
Time series modeling is uniquely challenged by the presence of autocorrelation in both historical and label sequences. Current research predominantly focuses on handling autocorrelation within the historical sequence but often neglects its presence in the label sequence. Specifically, emerging forecast models mainly conform to the direct forecast (DF) paradigm, generating multi-step forecasts under the assumption of conditional independence within the label sequence. This assumption disregards the inherent autocorrelation in the label sequence, thereby limiting the performance of DF-based models. In response to this gap, we introduce the Frequency-enhanced Direct Forecast (FreDF), which bypasses the complexity of label autocorrelation by learning to forecast in the frequency domain. Our experiments demonstrate that FreDF substantially outperforms existing state-of-the-art methods and is compatible with a variety of forecast models. Code is available at https://anonymous.4open.science/r/FreDF-0FB1.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Time series Long-term Forecast
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/f098078d0997928a7574fd1b547cf77e4a6a0830.pdf
learning on time series and dynamical systems
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Label Correlation Biases Direct Time Series Forecast
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission