Framer: Interactive Frame Interpolation
Video Frame Interpolation; Interactive; Diffusion Model; Correspondence Modeling
Yes
No ethics review needed.
- What are the differences in training / inference methodology between the autopilot and user-guided mode?
- Are there categories of videos where the results aren't as high quality as other categories?
- Could you explain a bit more about "considering that the conditions and the corresponding noisy latents are spatially aligned" near line 183?
- Can you explain how you transform 2D points into gaussian heatmaps? Are any other modalities like text used here? References to DragNUWA or DragAnything might be too recent. Consider also citing an older source such as from 2014 "Stacked Hourglass Networks for Human Pose Estimation" if it makes sense.
- Can the autopilot mode re-use the randomly sampled points ~line 197 instead of using feature matching to find new correspondences over time?
- Have you considered simpler alternatives for the Trajectory updating section?
- How do you handle camera motion in your framework?
The paper is mostly well written and easy to follow. The novelty here lies in the control branch for guiding the interpolation process that allows for fine-grained user-friendly control over the interpolation process.. The authors evaluate several existing methods on a variety of input types like real-world, sketch, cartoon, etc both qualitatively and quantitatively and produce high quality results.  Also included in a technique to produce the control signals automatically using bidirectional consistency checks. Several key design choices have been evaluated thoroughly, such as number of trajectories, bidirectional consistency.
The paper proposes a new technique for generating frame interpolations that can incorporate user input to drive the interpolation process between two given frames. The method uses Stable Video Diffusion as the base model with the inputs modified to include both the first and last frame. Additionally, a control branch consisting of gaussian heatmaps is encoded and included in the model. The method is evaluated on a variety of image categories and on publicly available datasets.
- There is no mention of inference speed, training speed or overall training methodology (end-to-end, multi stage, pretraining, etc)
- There appears to be no underlying camera model for handling the "novel view synthesis" examples. Having such a model could potentially improve quality while providing another user-control.
- The paper isn't very easy to follow specifically around the trajectory update for autopilot mode section.
Yes
No ethics review needed. Yes, Discrimination / bias / fairness concerns Yes, Privacy, security and safety Yes, Legal compliance (e.g., GDPR, copyright, terms of use) Yes, Potentially harmful insights, methodologies and applications Yes, Responsible research practice (e.g., human subjects, data release) Yes, Research integrity issues (e.g., plagiarism, dual submission) Yes, Unprofessional behaviors (e.g., unprofessional exchange between authors and reviewers) Yes, Other reasons (please specify below)
1)	Can the frame interpolation function support arbitrary number of frames generation between the start and end inputs?
2)	how to process the case when the number of corresponding points between frames is too few?
1）	The method is straight forward but effective, which can be used for many applications;
2）	Most results shown in this paper are impressive.
3）	Ablation experiments are sufficient to evaluate the method.
This paper presents an interactive frame interpolation framework to produce smoothly transitioning frames between two images, interactively or automatically. By harnessing user input point controls from the start and end frames, this method can effectively generate the video frames for different applications including drag control, novel view synthesis, cartoon and sketch interpolation, time-lapsing video generation, slow-motion video generation, and image morphing.
1)	The component of the Framer includes Trajectory preprocessing, Trajectory control and Interpolation. Although the method is effective, the novelty is limited. Trajectory control is designed by following DragNUWA, and the Interpolation is mainly based on SVD that determines the performance of this method.
2)	The examples shown in this paper are a bit simple. The motion and the difference between two key frames (first and last frames) are not complex. More complex scenarios, as cases shown in the paper “Training Weakly Supervised Video Frame Interpolation with Events”, should be considered to demonstrate the superiority of this method.
3)	The image quality of some results shown in this paper degrades compared to the first input frame. For example, the IQ of frames in “laugh.mp4” is worse than that of the start and end frames;
4)	The reconstruction metric of this method is not best when comparing it to related methods.
Yes
No ethics review needed.
Please address the issues raised in the weaknesses section. This reviewer is willing to increase the rating if the issues are well addressed.
The method’s introduction of keypoint trajectories addresses motion ambiguity, leveraging the stable video diffusion model to improve interpolation fidelity.   
The qualitative outcomes are notably strong.
This paper presents a diffusion-based architecture, *Framer*, for interactive frame interpolation, introducing last-frame conditioning and keypoint trajectories to achieve controllable transitions. While the qualitative results are impressive, the practical application, related work discussion, and unique technical contributions need further improvement.
1. **Related Work**: The paper does not sufficiently discuss previous work tackling motion ambiguity and interactive frame interpolation, such as [1] and [2], which address motion/velocity ambiguities. And [2] is the early work in interactive VFI. A more comprehensive discussion would clarify the novelty and contributions of *Framer*.
2. **Real-World Application**: The practical applicability of this method is unclear, especially given that real-world videos typically run at frame rates like 24–30fps, making extremely large-motion interpolation less common. There is also no analysis of inference speed in autopilot mode.
3. **Robustness and Scope**: The method’s effectiveness under complex trajectories remains uncertain, as the learned keypoint trajectory distribution is unlikely to generalize to all cases. Additionally, autopilot mode’s reliance on feature-matching algorithms may limit performance under conditions like blur or low resolution. Further experiments are needed to clarify performance under these situations.
4. **Benchmarking and Quantitative Analysis**: The paper lacks evaluations on commonly used VFI datasets, such as Vimeo90K and X4K1000FPS, which would offer valuable comparison. In addition, considering that the proposed method achieves superior performance only in terms of FVD, the inclusion of non-reference metrics such as NIQE will help to further understand the image quality.
5. **Technical Innovation**: While the method yields high-quality visuals, the technical novelty appears limited. Techniques such as conditioning diffusion models with start and end frames and using keypoint trajectories (DragGAN) already exist. Emphasizing unique contributions would strengthen the paper’s impact.

[1] "Exploring motion ambiguity and alignment for high-quality video frame interpolation." CVPR2023  
[2] "Clearer Frames, Anytime: Resolving Velocity Ambiguity in Video Frame Interpolation." ECCV2024.
Yes
No ethics review needed.
Overall, I think this is a solid piece of work. Aside from the minor points mentioned in the weaknesses section, I don’t have significant concerns. The approach is interesting and well-presented, with promising results. I would lean toward recommending acceptance of this paper.
1. The motivation of the paper is clear and well-founded. The authors address the inherent variability in intermediate frame generation and propose an effective solution by integrating keypoint control, which allows for a more tailored interpolation process that better aligns with user preferences.
2. The paper is well-written and easy to follow. The use of figures and tables is excellent, with ample illustrative examples demonstrating the model’s application.
3. The results presented are noticeably better than existing methods, particularly in challenging scenarios. Additionally, the inclusion of an automated mode ensures fairness in comparison, which further strengthens the validity of the results.
The paper proposes Framer, an interactive frame interpolation model designed to generate smooth frame transitions between two images. The approach allows users to customize the interpolation process by adjusting the motion trajectories of keypoints. This interactive feature provides precise control over local motions, improving the model’s ability to handle challenging scenarios. The model also includes an autopilot mode that automatically estimates keypoints and refines trajectories, simplifying the interpolation process for users.
1. It would be helpful to see examples of failure cases, if any, to better understand the model’s limitations. Displaying such cases could provide insight into potential improvements or areas where the method might struggle in real-world applications.
2. The title might be slightly misleading. The approach seems more aligned with image morphing rather than traditional frame interpolation. A title that reflects this focus on image-to-image transitions could better represent the paper’s core contribution.
We propose Framer, an interactive frame interpolation method that allows users to produce smoothly transitioning frames between two images by customizing the trajectory of selected keypoints, enhancing control and handling challenging cases.
@inproceedings{
anonymous2024framer,
title={Framer: Interactive Frame Interpolation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=Lp40Z40N07},
note={under review}
}
We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking the start and end frames as inputs, our approach supports customizing the transition process by tailoring the trajectory of some selected keypoints. Such a design enjoys two clear benefits. First, incorporating human interaction mitigates the issue arising from numerous possibilities of transforming one image to another, and in turn enables finer control of local motions. Second, as the most basic form of interaction, keypoints help establish the correspondence across frames, enhancing the model to handle challenging cases (e.g., objects on the start and end frames are of different shapes and styles). It is noteworthy that our system also offers an "autopilot" mode, where we introduce a module to estimate the keypoints and refine the trajectory automatically, to simplify the usage in practice. Extensive experimental results demonstrate the appealing performance of Framer on various applications, such as image morphing, time-lapse video generation, cartoon interpolation, etc. The code, the model, and the interface will be released to facilitate further research.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Video Frame Interpolation; Interactive; Diffusion Model; Correspondence Modeling
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/05e1b1f5544323d9a1b6b5782cc0c46e0cfa7b2b.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/a0e363e5d5672f581a8dbe1e393451a7fb0e7291.zip
Framer: Interactive Frame Interpolation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission