CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer
Weather and climate forecasting
Yes
No ethics review needed.
I am wondering why the proposed method patch all the different longitudinal areas into the same patch. What prevents the model from taking a 2d input HxWxD and do e.g. axial transformer (intervene between attention along H axis and then W axis) with Fourier Transform.
1. The experiments are extensive, solid, and the results are good. The paper compares the proposed method with both numerical method and ml method, and the metrics on latitude-weighted RMSE and Anomaly are strong and solid. The visualization graphs are very useful to understand the impact of the proposed method in different dimensions.

2. The proposed method well utilize the characteristics of the input data modality. The circular patching strategy introduce a effective inductive bias, and the DFT well utilize the latitudinal spatial periodicity.
This paper propose an improved approach to the Subseasonal-to-Seasonal (S2S) climate forecasting problem. The method mainly has two novel parts: 1. A better input patching method which utilize the circular nature of the graticule. 2.  Propose to intervene Discrete Fourier Transform throughout the transformer layers. 

The propose methods demonstrate improvement over all the existing methods, both numeric methods and data-driven method. And the propose method is especially effective in the long time horizon prediction compared to other autoregressive data-driven methods which suffer from error aggregation. The method also show better performance in the hard-to-predict areas like polar areas.
1. The comparison to existing method does not include computation complexity/model size. This weakens the comparison to other methods because the reader don't know whether the improvement is brought by the network architecture design or simply due to the increase in model size.

2. It would be a plus if the paper could also include a ablation study of autogressive prediction vs directly predicting all the future values, to ablate the effectiveness of the autoregressive formulation. But I understand this might be consuming.
Yes
No ethics review needed.
Add discussion with GraphCast.
1. The paper is well-written, and the idea and motivation are intuitive. Figure 1 clearly highlights the limitations in previous methods, providing a strong rationale for the proposed approach.

2. CIRT introduces significant innovations to address the lack of geometric inductive biases in prior work. Specifically, it partitions the graticule uniformly by latitude and leverages the Fourier transform to capture global features, which are then processed with self-attention.

3. The experimental results effectively demonstrate the superiority of CIRT, comparing favorably with both numerical and data-driven models.
CIRT propose a novel network for accurate S2S climate forecasting. The key idea is to model the cyclic characteristic of the graticule, consisting of two designs: (1) decomposing the weather data by latitude into circular patches; (2) Use fourier transform in self-attention. Extensive experiments on ERA5 demonstrate the effectiveness of the proposed CIRT network.
GraphCast employs a graph neural network for weather forecasting, sharing a similar initial motivation with CIRT in aiming to reduce inappropriate geometric inductive biases. The authors should consider providing a more in-depth discussion on the distinctions between GraphCast’s model design and system architecture compared to their own, highlighting the unique aspects and advantages of their final approach.
Yes
No ethics review needed.
1、The author should explain in detail how the results of the 1.5° experiment of the Pangu model were obtained. In global forecasts, there is currently a 0.25° prediction model. Is the 1.5° model necessary?
2、The experimental conditions of variables such as wind u and wind v.
3、Most weather models are tuned after training is complete, the article does not perform this work and fine-tuning can be used to further improve model performance.
The article has clear logic and ideas. The proposed method is very consistent with the research problem. It considers the problems faced by Patch Embedding in global meteorological modeling. Compared with most methods, it has achieved excellent results in high-latitude regions.
The article designs a circular transformer (CirT) to model medium—and long-term numerical forecasts. It takes into account the spherical structure of the ground, designs corresponding circular patches as input tokens of the Transformer, and uses the Fourier transform in self-attention to capture global information to model the spatial periodicity, achieving high accuracy.
1、The data in the experiment of the article is 1.5° instead of 0.25° as used in the Pangu model. Obviously, the author should give a detailed explanation of the experiment.
2、The article lacks experimental conditions for key variables such as wind u and wind v.
3、The article states that the usage of the Transformer network with Fourier transform is an important component, but both AFNO and Fourcast use related structures, which is not innovative enough.
@inproceedings{
anonymous2024cirt,
title={CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=YslOW2SO6S},
note={under review}
}
Accurate Subseasonal-to-Seasonal (S2S) climate forecasting is pivotal for decision-making including agriculture planning and disaster preparedness but is known to be challenging due to its chaotic nature. Although recent data-driven models have shown promising results, their performance is limited by inadequate consideration of geometric inductive biases. Usually, they treat the spherical weather data as planar images, resulting in an inaccurate representation of locations and spatial relations. In this work, we propose the geometric-inspired Circular Transformer (CirT) to model the cyclic characteristic of the graticule, consisting of two key designs: (1) Decomposing the weather data by latitude into circular patches that serve as input tokens to the Transformer; (2) Leveraging Fourier transform in self-attention to capture the global information and model the spatial periodicity. Extensive experiments on the Earth Reanalysis 5 (ERA5) reanalysis dataset demonstrate our model yields a significant improvement over the advanced data-driven models, including PanguWeather and GraphCast, as well as skillful ECMWF systems. Additionally, we empirically show the effectiveness of our model designs and high-quality prediction over spatial and temporal dimensions.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Weather and climate forecasting
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/66fd6c04a2bba9bdc7d14824cb37736b2bdde7f8.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission