Interactive Adjustment for Human Trajectory Prediction with Individual Feedback
Human Trajectory Prediction
Yes
No ethics review needed.
The most important question: How does this approach compare to prior works on adaptive trajectory prediction? Why is IAN better or more preferable to use by practitioners?

How does this approach perform on modern large-scale datasets with multiple types of agents and varying dynamics, e.g., Waymo Open Motion Dataset, nuPlan? These datasets also have less saturated metrics which should make it easier for the IAN's performance improvements to stand out.
The idea is sound and sensible.

The writing is clear and it is easy to follow the core idea being proposed.

There are a wide variety of baseline approaches used to demonstrate the core idea.
In this work, an online learning methodology is proposed for trajectory prediction based on the insight that ground truth observations checking past predictions (named feedback) can be collected as time goes on. To leverage this insight, the authors propose an interactive adjustment network (IAN) and evaluate its efficacy using a variety of trajectory prediction algorithms on a set of pedestrian and athlete motion datasets.
In contrast to the claims in the paper, this work is not the first to study feedback in trajectory predictions. Accordingly, the primary weakness of this work is a lack of discussion and comparisons to prior adaptive prediction work that investigates this same idea. Notable examples (which focus on both temporal adaptation, equivalent to individual feedback, and geographic adaptation) include:
* Y. Xu, L. Wang, Y. Wang, and Y. Fu, "Adaptive trajectory prediction
via transferable GNN," in IEEE Conf. on Computer Vision and Pattern
Recognition, 2022.
* B. Ivanovic, J. Harrison, and M. Pavone, "Expanding the deployment envelope of behavior prediction via adaptive meta-learning," in IEEE Conf. on Robotics and Automation, 2023.

The experimental setup relies primarily on small-scale pedestrian-only datasets. Further, these datasets are quite old (ETH/UCY are 10-15 years old, GCS is more than 10 years old, the NBA dataset is almost 10 years old), meaning performance has largely saturated on them and it is difficult to tell if the proposed method significantly improves upon baselines.

As stated in Section 5.4, a prediction model and the IAN are _consecutive_ modules, meaning they have to run one after the other (and not simultaneously as stated in Line 523). This means that there is a 20ms overall runtime increase as a result of adding the IAN.
Yes
No ethics review needed.
1. How do you justify the assumption that ground truth trajectories are available in real-time for computing feedback? In what practical scenarios would this be feasible?
2. How does your method differ from existing online learning approaches that adapt models based on prediction errors?
3. How does the computational cost of IAN affect inference time in real-world applications (even a small delay in prediction can go a long way)?
4. Are there experiments to isolate the effects of each component of your model? If so, please provide detailed results.
Statistical Significance: Are the reported improvements statistically significant? Can you include variance measures or confidence intervals?
5. How does your model handle agents for whom no feedback data is available?
- The concept of utilizing individual feedback from previous predictions is interesting.
- The proposed method is designed to be model-agnostic and can be applied to various trajectory prediction frameworks.
- Experiments are conducted on multiple datasets and with several baseline models.
The paper proposes an Interactive Adjustment Network (IAN) that leverages individual feedback from previous predictions to improve human trajectory prediction models. The idea is to use the differences between earlier predictions and actual trajectories (ground truths) to adjust future predictions for the same agent. The authors claim that their method can be applied as an external module to various existing prediction models and that it significantly boosts performance on datasets.
Major issues:
- The paper's biggest assumption is the fact is that IAN relies on having immediate access to the ground truth trajectories of agents to compute feedback. In real-world applications, example autonomous driving or robotics, such ground truth data is usually never available in real-time. 
- The concept of using feedback from previous predictions is not entirely new. Adaptive models and online learning techniques have long incorporated past errors to improve future predictions. The bigger issue is the lack of adequate differentiation of IAN from existing methods i.e., a thorough literature review that situates its contributions within the broader context.
- While the authors introduced a new loss function, there is no analysis of the properties of the displacement expectation loss or proofs of convergence and stability.
- There is a lot of poorly defined notation, making it difficult to understand the proposed method fully. It would make sense to move the table 3 from the appendix (which has the notation!) to the main paper. 

There are some issues with the presentation. 

Writing: 
- Section 3, Trajectory Prediction (just before equation 1): \hat{Y} of "him"
- Section 4.5, "At the beginning of this paragraph" 
- Section 5.4, Last sentence: Our approach can "inference" at a high frequency
- Personal preference: please use "Figure X represents ..." (not abbreviation) instead of "Fig. X represents..." because your captions have the former. 

Images:
- The image labels for Figure 2 are too small to read -- had to Zoom 150% to read it
- Figure 2, it is hard to tell the difference between Train Path and Test Path, the arrows look very similar, please consider a different color. 
- Figure 4, 5, it is hard to see lighter colors.
Yes
No ethics review needed.
1. How is the feedback being connected (with A_F for example) in Eq3-4 and Eq9-10, and being differentiated between training and testing stage?
1. Novel design for trajectory prediction pipeline considering closed-loop decoding (or individual feedback). Through proposed feedback generater and adjuster model in IAN, the multi-modal prediction garners extra feedback consistency supervisions in prediction confidence.

2. Comprehensive experimental evaluations. 1) Broad improvements are reported across various datasets when integrating IAN with several SOTA motion prediction frameworks; 2) Clear comparison with other decoding strategies.
In this paper, an individual feedback framework (termed IAN) for Trajectory Prediction is proposed that dynamically adjusts the confidence score conditioned on prediction consistency between prediction and GT trajectories. Through addtional feedback generator during training, a consistency score could be derived through attention in supervising the prediction condifence. Comprehensive experiment results demonstrate the effectiveness of IAN module when collaborating with a series of state-of-the-art predictors.
1. Unclear writings in methodology: It is partly clear to understand the feedback mechanism by Figure 2, However, by simply go through the content the reviewer could hardly understand the feedback generation process between training and testing. Hence, it is better to provide additional Algorithm part for clearer understanding. 

2. Additional evaluation for the feedback. In Tab1-2, minADE/minFDE is a direct metric in mearusing the best-case similarity for multi-modal predicted trajectory. However, it is unclear whether the confidence are well-performed by the feedback enhancement. Hence, several extra results measured by Brier-minADE/FDE, MissRate, or mAP seems needed.
Yes
No ethics review needed.
However, this paper provides a promising way to improve the trajectory prediction method by incorporating the agent-specific past trajectories and groundtruths, here are some comments that need to be addressed
1. Since the authors introduced the individual feedback in the paper and provided the intuition, it would be good to see any theoretical justification for why it is important.
2. It would be good to see if the authors can address that incorporating the agent-specific previous trajectories may not lead to overfitting the base-model prediction. 
3. Another important concern is that, since the IAN is heavily dependent on the base-model prediction if there are accumulated errors in the base-model prediction, then the proposal generated from the IAN would also be spurious. What kind of risk assessment should be incorporated into the IAN?
4. It would be good to see if there is more explanation on the architecture of IAN, including the feature generation network, what kind of encoder architecture is used, and the rationale behind it.
5. Another thing that would be good is for the authors to include at least some description of the process happening in the figures to help the reader understand the figures.
6. In the conclusion, the authors should include the limitations of their work and possible future directions. It would also be good to see if authors can report the failure cases.
Following are the strengths of this paper:
1. The introduction of individual feedback to enhance trajectory prediction by integrating agent-specific past trajectories with groundtruth is promising.
2. Evaluation of the proposal IAN network on trajectory prediction benchmarks highlights their claim.
3. As an external module, IAN can be integrated with various trajectory prediction methods.
4. The presentation of results is good.
This paper introduces the concept of individual feedback in trajectory prediction problems by designing an Interaction Adjustment Network (IAN). The IAN, comprising a feedback generator, an adjuster, and a filtering process, is designed to be an external module that can be seamlessly integrated with other trajectory prediction models. To ensure its adaptability, the authors have designed a displacement loss function to train the IAN. Experiments have shown the IAN's efficacy on widely adopted benchmarks for trajectory prediction.
However, this paper provides a promising way to improve the trajectory prediction method by incorporating the agent-specific past trajectories and groundtruths, here are some comments that need to be addressed
1. Since the authors introduced the individual feedback in the paper and provided the intuition, it would be good to see any theoretical justification for why it is important.
2. It would be good to see if the authors can address that incorporating the agent-specific previous trajectories may not lead to overfitting the base-model prediction. 
3. Another important concern is that, since the IAN is heavily dependent on the base-model prediction if there are accumulated errors in the base-model prediction, then the proposal generated from the IAN would also be spurious. What kind of risk assessment should be incorporated into the IAN?
4. It would be good to see if there is more explanation on the architecture of IAN, including the feature generation network, what kind of encoder architecture is used, and the rationale behind it.
5. Another thing that would be good is for the authors to include at least some description of the process happening in the figures to help the reader understand the figures.
6. In the conclusion, the authors should include the limitations of their work and possible future directions. It would also be good to see if authors can report the failure cases.
For the first time, we study individual feedback, which is derived from early predictions on a target, to improve the accuracy of subsequent predictions on the same agent. We design an external module IAN to model it and prove its effectiveness.
@inproceedings{
anonymous2024interactive,
title={Interactive Adjustment for Human Trajectory Prediction with Individual Feedback},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=DCpukR83sw},
note={under review}
}
Human trajectory prediction is fundamental for autonomous driving and service robot. The research community has studied various important aspects of this task and made remarkable progress recently. However, there is an essential perspective which is not well exploited in previous research all along, namely individual feedback. Individual feedback exists in the sequential nature of trajectory prediction, where earlier predictions of a target can be verified over time by his ground-truth trajectories to obtain feedback which provides valuable experience for subsequent predictions on the same agent. In this paper, we show such feedback can reveal the strengths and weaknesses of the model's predictions on a specific target and heuristically guide to deliver better predictions on him. We present an interactive adjustment network to effectively model and leverage the feedback. This network first exploits the feedback from previous predictions to dynamically generate an adjuster which then interactively makes appropriate adjustments to current predictions for more accurate ones. We raise a novel displacement expectation loss to train this interactive architecture. Through experiments on representative prediction methods and widely-used benchmarks, we demonstrate the great value of individual feedback and the superior effectiveness of proposed interactive adjustment network. Our code will be made publicly available.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Human Trajectory Prediction
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ae84be591bcca033f0b18e32c111cb415afd51f9.pdf
applications to robotics, autonomy, planning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/cae6d843d5d4ad215a2bcec50e89800d8f138deb.zip
Interactive Adjustment for Human Trajectory Prediction with Individual Feedback
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission