Zero-shot Imputation with Foundation Inference Models for Dynamical Systems
Zero-shot imputation;foundation models;time series imputation;dynamical systems;amortized inference;zero-shot interpolation;foundation models for time series
Yes
No ethics review needed.
— The FIM-l model only works for time series that follow a ‘simple’ distribution. Since existing imputation models already perform well on real-world and synthetic data, the practical value of the proposed model might be limited.
— The authors did not explain clearly how the FIM could handle processes of any dimensionality, and how the performance of the model varies with the dimensionality in practice.
— The proposed FIM has great zero-shot imputation performance and achieves SOTA results on multiple datasets.
— Unlike prior work, this paper proposes a zero-shot approach that can be used on processes of any dimensionality.
— The synthetic data generation model offers a general method for creating meaningful synthetic time series, which could support future research.
- The authors are honest about the limitation of the approach, especially those related to the synthetic distributions used in the analysis and setting (ODEs) as opposed to real-world data which can exhibit any (or no ) distrubution.
This paper proposes a supervised learning framework for zero-shot time series imputation in dynamical systems of any dimensionality. The framework uses a synthetic data generation model based on two key assumptions: 
(1) time series with point-wise missing data have simple ODE-based interpolation solutions, and 
(2) time series with temporal missing patterns are locally simple. 
The authors introduce two neural interpolation models, FIM and FIM-l, designed for point-wise and temporal missing patterns, respectively.
The approach is evaulated over 8 datasets
- Fig 2 is difficult to read, the colors are too  similar
- The paper is too "buzzwordy", I would have  preferred more of a proper technical discussion
- The results of the proposed method are not always close to SOTA or better than SOTA
Yes
No ethics review needed.
- I think some ablation studies for measuring the generalization capability of this model will be beneficial.
- This paper is well written and organized. I enjoy reading this paper.
- To the best of my knowledge, this paper is novel in that this firstly introduces the zero-shot imputation of time-series with dimension free inference. 
- I think the training objective and the way they give supervision to the model is also novel.
- The experimental results are impressive, covering a wide range of datasets and scenarios enough to prove the zero-shot capability of this method. Additionally, the paper faithfully includes details for reproducing the experiments which is helpful
This paper introduces foundation model for time-series imputation, using several parametric functions of ODEs for training the model. The authors generate many synthetic noisy, irregular observations to train recognition model in offline manner.  They use combination of objective function with respect to initial value and data points to train this model. This trained model can conduct zero-shot imputation on unseen time series dataset with different dimensions. They validate their method on various different benchmarks in several domains.
- Although this model impute the data in zero-shot manner, experimental results are not that impressive in Table 2. Also, the authors use reported value from the other papers which can question whether comparison is really fair.
-  I think it would be better to include ODE based methods for the baseline considering concept of this paper.
Yes
No ethics review needed.
see above.
1. very interesting work on time series imputation, potentially pointing out a new direction
2. design of neural recognition model is novel and interesting, albeit inspired by deepOnet. Such "borrowing" is still interesting. 
3. zero-shot performance is surprisingly good
The paper develops a ODE-based foundation model for time series imputation. The idea is to simulate a large collection of incomplete time series including both point-wise and temporal missing patterns. A recognition model is trained to map such time series to the initial condition, and time derivatives at each time step, with both mean and log-variance prediction. The design of the recognition model is inspired by deep ONet. There is no need to fine tune the model on application-specific data. The experiments show the improvement over many sota methods specifically trained on application data.
1. as admitted by the authors, one limitation is that when the actual application does not well match the assumption of the simulation data, the performance can deteriorate. Though the paper focuses on zero-shot prediction, it will be great to discuss  possible methods for fine tuning or adaptation to specific applications. Since your training assumes the ground-truth of the initial and time derivatives of each sampled trajectories are known, and the neural recognition models directly learn to fit them, such training cannot directly apply to real dataset where these ground-truth is unknown. 
2.  The design of the neural recognition model seems not well justified, especially for the branch-net component. why do you use RNN? Why not using attention mechanism instead? Given RNN/LSTM's are replaced by attention  nearly every where, the authors should explain their rationale of such choices, either theoretically or empirically. 
3. How are the hyperparameters selected? Why using 1024 dimensional embeddings? Are this done by a rigorous validation process? If so, how?  It will be good for the authors to specify the details of hyperparmeter selection and validation process.
Yes
No ethics review needed.
Please find the questions in the Weaknesses section above.
1. The problem of imputing time-series data is important and the idea of linking amortized inference and neural operator is interesting.
2. The methodology of the foundation inference model is well-structured.
3. Experiments were performed on the imputation of point-wise and temporal missing patterns to present the effectiveness of the proposed method.
This paper addresses the challenges of imputing missing time-series data. A framework for zero-shot time-series imputation is proposed with 1) a synthetic data generation model for sampling a set of ODE solutions and 2) a neural recognition model mapping the time series data onto parametric functions. Empirical results present improved performance of imputation compared to baseline models.
1. I think the idea of zero-shot amortized inference framework is somewhat incremental. There have been works on amortized inference for few-shot time-series forecasting (e.g. [1] [2] [3][4]) in terms of lacking sufficient observation and learning single dynamics, where the idea of amortized inference learning the prior knowledge of dynamics is similar to the proposed method. Could the authors add a discussion about these works and the benefits of the proposed method? Also, it would be good if the authors could make a comparison of some of these methods in experiments.
2. In Equation 3, 4, and 5, do $\phi^{\theta}_i$ and $\psi^{\theta}$ share the same parameters $\theta$? Similar problem to the parameter $\psi$ in FIM model in Equation 7 and 8.
We introduce a framework for zero-shot imputation of missing time series data, whose underlying dynamics are assumed to be determined by ODEs. Our foundation models often outperform state-of-the-art methods, which are trained on the target datasets.
@inproceedings{
anonymous2024zeroshot,
title={Zero-shot Imputation with Foundation Inference Models for Dynamical Systems},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=NPSZ7V1CCY},
note={under review}
}
Dynamical systems governed by ordinary differential equations (ODEs) serve as models for a vast number of natural and social phenomena. In this work, we offer a fresh perspective on the classical problem of imputing missing time series data, whose underlying dynamics are assumed to be determined by ODEs. Specifically, we revisit ideas from amortized inference and neural operators, and propose a novel supervised learning framework for *zero-shot time series imputation*, through parametric functions satisfying some (hidden) ODEs. Our proposal consists of two components. First, a broad probability distribution over the space of ODE solutions, observation times and noise mechanisms, with which we generate a large, synthetic dataset of (hidden) ODE solutions, along with their noisy and sparse observations. Second, a neural recognition model that is trained *offline*, to map the generated time series onto the spaces of initial conditions and time derivatives of the (hidden) ODE solutions, which we then integrate to impute the missing data. We empirically demonstrate that *one and the same* (pretrained) recognition model can perform zero-shot imputation across 63 distinct time series with missing values, each sampled from widely different dynamical systems. Likewise, we demonstrate that it can perform zero-shot imputation of missing high-dimensional data in 10 vastly different settings, spanning human motion, air quality, traffic and electricity studies, as well as Navier-Stokes simulations — *without requiring any fine-tuning*. What is more, our proposal often outperforms state-of-the-art methods, which are trained on the target datasets.

Our pretrained model is available with the supplementary material
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Zero-shot imputation foundation models time series imputation dynamical systems amortized inference zero-shot interpolation foundation models for time series
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/18163ca26e88640461b6cdd84dda548aecccf224.pdf
learning on time series and dynamical systems
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Zero-shot Imputation with Foundation Inference Models for Dynamical Systems
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission