Predicting the Energy Landscape of Stochastic Dynamical System via Physics-informed Self-supervised Learning
dynamical system;energy landscape;deep learning
Yes
No ethics review needed.
* Please discuss the issue of transferrability. Can the trained model be transferred to unseen structures? Was a separate model trained for each protein? Which of the sub-modules, encoder, decoder, etc, can be reused?

* Scalability. To model a bigger protein, how large does the codebook need to be? What about training and inference cost for longer sequences?

* Spell out KNN (k-nearest neighbor?)

* How are encoder Ξ, decoder Ω implemented? GCNN was mentioned but I still have no idea what's in the model.

* There is no ablation tests, so it's hard to see what the various loss terms or model designs offer. 

* There is no discussion of the limitations of the method.

* Consider adding some explanation of the graph FPE method. FPE models the evolution of the probability p, not its encoding Phi(p) as in eq (4). I suppose it is explained in the cited reference. But since it is a central piece of this paper, please explain how this works.
The idea of bringing together several novel methods for modeling stochastic dynamical systems is very interesting. Three different examples cover different application areas and add to the credibility of the method. The capability to learn unknown energy landscape from stochastic dynamical systems is a potentially high-impact contribution.
This paper proposes to infer the energy landscape of complex, stochastic dynamical systems. A distinguishing feature is the adoption of vector quantization techniques for dimension reduction. Dynamical information was inferred via a graph neural Fokker-Planck equation method. A regularization term was introduced to constrain the long-term prediction behavior between the empirical distribution p and the Boltzmann distribution. The novelty of bringing together these ideas is appreciated. Experiments were performed on 3 tests: 2d Prinz potential, ecological evolution, and fast-folding peptides. The method part of the paper is not very clear and somewhat difficult to follow. There is no ablation tests, so it's hard to see what the various loss terms or model designs offer. There is no discussion of the limitations of the method.
* The description of the overall model is not clear to me. At least not easy to follow. For example, the encoder, decoder, Phi, Psi, Xi,... were defined in different places and some were not clearly defined. What space and dimension does Phi map from and to? Please make it easier to follow. And the dimension of latents such as H is not clear. Please collect the equations in a coherent way, e.g. all the different losses, and what are the losses sampled over?

* Overall, the writing of the paper is too hand-waving. As a referee I don't like guesswork.
Yes
No ethics review needed.
* to address (3) above, please provide better justifications for the main model and the associated training losses

* to address (4) and (5), please compare with published results on recovering the energy and fitting new trajectories of new molecules or systems
This is an interesting learning problem, also requiring to learn a latent explanatory space for sampled continuous time trajectories. The authors compare against multiple baselines across everal environments. The results appear encouraging.
The authors propose an algorithm for discovering the energy function that may explain data trajectories assumed to be driven by the gradient of the energy plus noise. Their model learns a partitioning of the space with one discrete symbol per region and an associated energy value, along with a decoder (and associated encoder) mapping to the observed continuous space. They assume that the dynamics is driven by the energy difference between nearby partitions and Markovian transitions between regions. The model is evaluated on three tasks and compared with several baselines.
(1) I do not see how such an approach can generalize outside of the visited regions, i.e., the cluster centers corresponding to the codebook entries; in general, the most interesting energy functions can have a very large (if not exponential) number of energy wells, and one needs to generalize from the visited wells to new ones. The reason is that many factors may be composed that give rise to stable solutions. Hence a multivariate representation of the discrete identities must be constructed (think about language, which is discrete but allows generating a huge number of legal combinations of words).

(2) It is not clear to me that g in general will preserve all information about the energy of the state, so it may be necessary for the encoder (from observations to states) to look at the whole past trajectory in order to make a probabilistic guess about the state proxy.

(3) the main model (sec 3.2) is not sufficiently understandable and could use more motivation for its different parts. Why these equations in particular (4 and 5).

(4) The experimental setup seems to lack comparisons with *published results* from earlier work, which makes it difficult to know if the comparisons are fair.

(5) The test set includes only trajectories from the same system for which a training trajectory is given, i.e., there is no form of generalization to new systems (e.g., new molecules).
Yes
No ethics review needed.
See Weaknesses
The paper begins with a strong motivation, highlighting the challenges in obtaining direct energy values. Additionally, it provides a solid rationale for incorporating discrete codebook embeddings into their method, based on the inherently low dimensionality of energy landscapes.
Estimating the energy landscape is challenging because obtaining direct energy measurements is costly. To bypass this process, the paper proposes a physics-informed self-supervised learning approach, where the energy landscape is estimated from historical evolution trajectories instead of direct energy signals. This method uses discrete codebook embeddings, assuming that energy landscapes generally have low intrinsic dimensionality. Additionally, a physics-informed graph neural Fokker-Planck architecture and physics-inspired regularization are employed to predict system states more accurately.
1. The ablation studies in this paper seem too limited. Since multiple modules are introduced in their method, additional experiments are needed to clarify the individual contributions of each component to the overall performance. For instance, it would be helpful to examine the effect of using discrete versus continuous embeddings. Similarly, given that five types of losses are included, an analysis of each loss’s specific impact would provide valuable insights. If these results were included, I would increase my rating.

2. Regarding  $L_{phy}$, could you explain in more detail why this loss is referred to as “physics-inspired”?

3. here are five types of losses in total. Could you clarify the exact meaning of the terms $\mathbf{p}$ and $\mathbf{q}$ in each loss?
Yes
No ethics review needed.
- What is the computational complexity of the proposed methods and its comparison to existing methods? 

- Are the estimate energy landscape consistent under different hyperparameters?

- Does the granularity of the discretization of the energy landscape influence the accuracy of system evolution estimation?

- Considering that real-world trajectory data is usually noisy and sparse, how robust is the method to such data?
- The paper is clearly written. 

- The method is trained in a self-supervised manner, hence it can conduct estimation without knowing true energy landscape.

- The methods were evaluated on three interdisciplinary systems and all demonstrated an improvement compared with SOTA baselines.
This paper proposed a new framework PESLA to estimate the energy landscapes from the evolution trajectories of the system. The framework includes two stage, first it uses an adaptive codebook to obtain a discrete landscape space from the observation trajectories. Second, it uses a graph neural ODE based on the Fokker-Planck equation to predict system evolution.
- The author mentioned that “observable evolutionary trajectories typically cover only a limited portion of the vat state space”, however, the paper did not explain how the purposed method perform in the situation when the available data is biased or very sparse.

- The method relies on the assumption that the system is driven by energy and a well-defined low dimensionality energy landscape exists. Therefore, the assumption might limit the applicability, which was not discussed in the paper.

- The method is lack of interpretablity. The training fully relies on the final trajectory prediction errors to avoid knowing true energy landscape, however, it is not discussed how reliable or meaningful of the predicted trajectory and the estimated energy landscape. For example, dose the correlation of the estimated energy landscape and true energy landscape influence the final trajectory prediction?
We propose a self-supervised learning framework to predict the energy landscape of stochastic dynamical systems without supervisory signals for energy, while collaboratively achieving accurate energy estimation and evolution prediction.
@inproceedings{
anonymous2024predicting,
title={Predicting the Energy Landscape of Stochastic Dynamical System via  Physics-informed Self-supervised Learning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=PxRATSTDlS},
note={under review}
}
Energy landscapes play a crucial role in shaping dynamics of many real-world complex systems. System evolution is often modeled as particles moving on a landscape under the combined effect of energy-driven drift and noise-induced diffusion, where the energy governs the long-term motion of the particles. Estimating the energy landscape of a system has been a longstanding interdisciplinary challenge, hindered by the high operational costs or the difficulty of obtaining supervisory signals. Therefore, the question of how to infer the energy landscape in the absence of true energy values is critical. In this paper, we propose a physics-informed self-supervised learning method to learn the energy landscape from the evolution trajectories of the system. It first maps the system state from the observation space to a discrete landscape space by an adaptive codebook, and then explicitly integrates energy into the graph neural Fokker-Planck equation, enabling the joint learning of energy estimation and evolution prediction. Experimental results across interdisciplinary systems demonstrate that our estimated energy has a correlation coefficient above 0.9 with the ground truth, and evolution prediction accuracy exceeds the baseline by an average of 17.65\%. The code is available in the anonymous repository: https://anonymous.4open.science/r/PESLA-0D9A/README.md
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
dynamical system energy landscape deep learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/3b02f5b646d4dedb0b001622d52fb80c00854087.pdf
applications to physical sciences (physics, chemistry, biology, etc.)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Predicting the Energy Landscape of Stochastic Dynamical System via  Physics-informed Self-supervised Learning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission