Investigating Pattern Neurons in Urban Time Series Forecasting
urban time series forecasting;neuron detection
Yes
No ethics review needed.
- In line2 Algorithm1, why do author filter out low-frequency samples?
- Can we allocate low-frequency patterns by categorizing neuron activations? 
- Is the framework able to handle two or more low-frequency patterns?
- Improving forecasting performance on low-frequency events is interesting and challenging.
- The idea of pattern neuron is intuitive and reasonable.
- The proposed framework outperforms existing works.
This paper proposes PN-Train, which detect and fine-tune pattern neurons to improve the forecasting performance on low-frequency patterns.
- The framework requires explicitly defined low-frequency patterns, i.e., holidays, and a dataset with low-frequency patterns is also required (as in Eq(3)). It is unclear whether the framework could handle implicit low-frequency patterns.
- In PNV, it will be better to test whether removing PNs leads to significant performance decreasing on high-frequency patterns, since PNs could be critical for all patterns rather than just low-frequency patterns.
- Minor errors. For example, in Fig.2, PNV seems to deactivate neurons which are not pattern neurons.
Yes
No ethics review needed.
Refer to weaknesses
(1) The work proposes a perturbation-based detector that effectively identifies pattern neurons associated with low-frequency patterns.

(2) It designs a fine-tuning mechanism to enhance these pattern neurons without compromising the representation of high-frequency patterns. This design ensures high accuracy for both low-frequency and high-frequency events.

(3) The study provides empirical results that verify the effectiveness of the proposed method, demonstrating significant performance improvements.
This work introduces PN-Train, a novel urban time series forecasting method that significantly improves prediction accuracy by identifying and fine-tuning pattern neurons associated with low-frequency events, such as holidays. PN-Train employs a perturbation-based detector to recognize these neurons and enhances them through a fine-tuning mechanism without compromising the representation learning of high-frequency patterns. Empirical results demonstrate that PN-Train considerably boosts forecasting accuracy for low-frequency events while maintaining high performance for high-frequency events.
(1) The distinction between low-frequency and high-frequency events is primarily between holidays and non-holidays, which may not have substantial practical significance. Other baselines also improve accuracy for both low-frequency and high-frequency events simultaneously. Focusing on anomaly detection or traffic prediction under anomalous events might have been more meaningful.

(2) The absence of open-source code makes it difficult to reproduce the results.

(3) The study utilizes too few datasets. It would have been better to use datasets from the PeMS system, such as Large-ST [1], which are well-processed and cover longer time spans.

[1] Liu X., Xia Y., Liang Y., et al. LARGEST: A Benchmark Dataset for Large-Scale Traffic Forecasting. Advances in Neural Information Processing Systems, 2024, 36.
Yes
No ethics review needed.
Q1. Is there any way to quantitatively define or model the frequency of events? 

Q2. Is there any theoretical principle that can support the neuron-based strategy?
[1] A simple yet effective method is proposed to deal with the low-frequency patterns in urban time series data. It is of certain novelty to design a method from the neural aspect, making  

[2] Experiments on multiple datasets validate the existence of pattern neurons and the proposed PND stage successfully detects them. Overall, the empirical studies are solid. 

[3] The paper is well-written and easy to follow, with sufficient technical details on the pattern neuron detector, pattern neuron verifier, and pattern neuron optimizer.
This paper firstly investigates how urban time series models (UTSMs) can be utilized to deal with the low-frequence events in urban management. It proposes a pattern neuron-guided training (PN-Train) method consisting of a perturbation-based detector and a fine-tuning mechanism. The experiments on a series of time series forecasting tasks show the superiority of PN-Train, which enhances performance in low-frequency events while maintaining high performance in high-frequency events.
[1] It is better to have a more complete discussion about low-frequency events, e.g., how these events are handled in previous works (Related Works section), how low-frequency events should be technically defined, and how the evaluation metrics are established to verify the effectiveness of the methods. 

[2] Lack of comparison with baseline methods that deal with various unbalanced data distribution problems.
Yes
No ethics review needed.
Please see W1-W3.
S1: PN-Train introduces a novel approach by using a perturbation-based detector to identify neurons associated with low-frequency patterns and fine-tuning these neurons to improve prediction accuracy. This is an innovative perspective, combining neuron-level analysis with practical forecasting tasks.

S2: By capturing and fine-tuning neurons under different patterns, PN-Train enhances the fairness of predictions for low-frequency events, reduces model bias, and helps in forecasting scenarios with sparse data.

S3: The experiments provide solid evidence of the existence of Pattern Neurons, with case studies offering visualized results that make the concept highly intuitive and easy to understand.
This work addresses the challenge of predicting low-frequency events (e.g., holidays, emergencies) in urban time series data, which are often overlooked by traditional models. The authors propose Pattern Neuron guided Training (PN-Train), a novel method that identifies and fine-tunes neurons specifically associated with low-frequency patterns. PN-Train incorporates a perturbation-based Pattern Neuron Detector (PND) to locate key neurons, and a Pattern Neuron Optimizer (PNO) to enhance their learning without compromising high-frequency event predictions.
W1: While the paper introduces neuron detection and optimization methods, it lacks in-depth theoretical explanation for why these neurons exhibit distinct behavior for low-frequency events. More analysis and discussion[1,2] on the working mechanisms of pattern neurons, such as potential differences in weight distribution or activation between high- and low-frequency events, is needed, along with supporting theoretical calculations.

W2: The paper highlights patterns for holidays and emergencies in the dataset, but in real-world applications, low-frequency events can be more random and harder to predict. Further discussion on how PN-Train handles unpredictable low-frequency events, such as concert events, sport events, or extreme weather like typhoon,  and then testing its generalization ability in more complex scenarios would be beneficial.

W3: There are some typos, such as inconsistent citation formatting in reference 766, the use of "$B" in Figure 5b, and inconsistency between "B" in Figure 5 and "D" in line 486. Please clearfully check for typos throughout the paper.


[1] Zhou Z, Huang Q, Wang B, et al. ComS2T: A complementary spatiotemporal learning system for data-adaptive model evolution[J]. arXiv preprint arXiv:2403.01738, 2024.

[2] Tang T, Luo W, Huang H, et al. Language-specific neurons: The key to multilingual capabilities in large language models. ACL, 2024
@inproceedings{
anonymous2024investigating,
title={Investigating Pattern Neurons in Urban Time Series Forecasting},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=a9vey6B54y},
note={under review}
}
Urban time series forecasting is crucial for smart city development and is key to sustainable urban management. Although urban time series models (UTSMs) are effective in general forecasting, they often overlook low-frequency events, such as emergencies and holidays, leading to degraded performance in practical applications. In this paper, we first investigate how UTSMs handle these infrequent patterns from a neural perspective. Based on our findings, we propose $\textbf{P}$attern $\textbf{N}$euron guided $\textbf{Train}$ing ($\texttt{PN-Train}$), a novel training method that features (i) a $\textit{perturbation-based detector}$ to identify neurons responsible for low-frequency patterns in UTSMs, and (ii) a $\textit{fine-tuning mechanism}$ that enhances these neurons without compromising representation learning on high-frequency patterns. Empirical results demonstrate that $\texttt{PN-Train}$ considerably improves forecasting accuracy for low-frequency events while maintaining high performance for high-frequency events.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
urban time series forecasting neuron detection
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/4b3cdf08b68e3af3cc9d903b87d20483f32e202f.pdf
learning on time series and dynamical systems
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Investigating Pattern Neurons in Urban Time Series Forecasting
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission