Modeling Spatiotemporal Heterogeneity in Earth Science Machine Learning: An End-to-End Approach
AI4Science;Remote Sensing;Tabular Machine Learning;Spatiotemporal Prediction;Spatiotemporal Heterogeneity
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
I am pleased to see that you are willing to put in the effort to refine this paper, but the acknowledged issues are inherent. Completely changing many method setups would be unfair to other authors. Therefore, I would like to maintain the original score and increase the confidence level. I wish you good luck.
We appreciate the reviewer's recognition of our core work and their constructive suggestions for improvement. We will submit a revised version addressing all concerns before the rebuttal period ends.
Thank you for your thoughtful comment regarding the end-to-end nature of our framework. We would like to clarify that Node2Vec is not actually essential to our method. We had verified that initializing node embeddings with a standard normal distribution plus RoPE positional encoding in earlier experiment that could achieve similar accuracy levels as reported in our paper, albeit requiring longer training time (approximately 18 hours).

If you maintain that using Node2Vec constitutes a form of pre-training that violates true end-to-end training principles, we are willing to revise our paper to use the standard initialization approach without Node2Vec across all experiments. We believe this modification would align with your definition of end-to-end training.

We appreciate your feedback as it helps us improve the clarity and technical accuracy of our work.
I have reviewed all the remaining responses, and overall, the authors have admitted to most of the issues I pointed out. Although they promise to make improvements in the future, there are too many aspects that need revision. While I admire the authors' hard work, I believe that now is not the right time to publish this still immature paper.
Summary
Alright, I agree with this part of the response, it is very straightforward and engaging.
Well, the authors acknowledge that they first need to perform a predefined graph processing operation and a pre-training step for the nodes (implemented by the node2vec). The former can be seen as data engineering, while the latter requires separate training, although it might be lightweight. Based on these, the authors then train the actual model. However, this cannot truly be called end-to-end training, because the learning of the latter and the former is not coupled (of course, I know this is a common operational paradigm from 2015-2019, conceptually similar to pre-training + fine-tuning). I really can't understand why they would call their framework end-to-end.
**W3,W4:** While we acknowledge the need to include additional citations regarding spatio-temporal weight initialization and linear attention in our revised version, it's important to note that **our work addresses a fundamentally different problem from traditional spatio-temporal prediction tasks**. Our focus is on **tabular machine learning with spatio-temporal characteristics**, where each row in the table represents data from different locations and timestamps. This is distinct from conventional spatio-temporal learning methods that typically deal with grid-like or graph-structured data.

The core contribution of our paper lies in how we **reformulate the challenging problem of modeling non-stationary relationships in tabular Earth science data**, where the associations between features and targets vary across both space and time dimensions. While we utilize some existing technical components as implementation tools, our **main innovation is in the novel problem formulation and solution framework** for handling such complex tabular data structures.

**W5:** Regarding the decoder details, we mentioned in the paper that we used the **standard PyTorch transformer decoder implementation**, which is a widely-used module in the field. Given its widespread adoption and standardization, we didn't see the need for extensive explanation in the main text.

**W6:** As for the efficiency-related experiments, please refer to our response to Q4.
Reply on Weakness - 3.4.5.6
We appreciate the reviewer's concerns regarding computational complexity. Our implementation employs a local weight sharing strategy with 750 cluster centers for graph construction. This means that spatial points within small regions generated by K-means share the same output embedding for decoding. With only 750 nodes and approximately 50 edges per node, the node2vec computation does not introduce significant computational overhead and typically completes within minutes.

Concerning the end-to-end nature of our approach: While we understand the reviewer's concern about using node2vec initialization, we would like to clarify that:

- node2vec+RoPE is only used to calculate the initial node embeddings to ensure training stability
- Both the node embeddings and edge weights are defined as `nn.Parameters` with `require_grad=True`
- These parameters are further optimized during backpropagation
- The initialization merely provides a starting point, after which the model learns and adjusts these parameters end-to-end through gradient descent

This implementation allows us to maintain the benefits of end-to-end learning while leveraging node2vec's structural insights for better initialization, ultimately contributing to more stable and effective training.

We welcome any further discussion regarding the learning of these parameters if the reviewer has additional concerns.
Reply on Weaknesses - 2
Our primary objective was to construct a graph with cyclic structure, so we initially built this graph under an idealized spherical model to calculate initial distance weights, specifically to validate the effectiveness of our cyclic graph construction approach. Regarding the computational complexity concerns, we do not treat each spatial point as an individual graph node. Instead, we employ clustering to merge adjacent spatial points, ultimately constructing the graph on 750 spatial points. This approach maintains reasonable computational demands, typically completing within a few minutes. 

Finally, we acknowledge the reviewer's suggestion about using an ellipsoidal coordinate system is valid, and we plan to explore this approach and update our paper accordingly.
Reply on Weaknesses - 1