SeqAfford: Sequential 3D affordance reasoning via Multimodal Large Language Model
Multimodal Large Language Model;3D Affordance Reasoning
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
This paper introduces Sequential 3D Affordance Reasoning, a task to handle complex user instructions; and presents the SeqAfford model and a 180K instruction-point cloud dataset for enhanced affordance segmentation and reasoning.
@misc{
wang2024seqafford,
title={SeqAfford: Sequential 3D affordance reasoning via Multimodal Large Language Model},
author={Hanqing Wang and Chunlin Yu and Haoyang Luo and Jingyi Yu and Ye Shi and Jingya Wang},
year={2024},
url={https://openreview.net/forum?id=pL31n7F6Ro}
}
3D affordance segmentation aims to link human instructions to touchable regions of 3D objects for embodied manipulations. Existing efforts typically adhere to single-object, single-affordance paradigms, where each affordance type or explicit instruction strictly corresponds to a specific affordance region and are unable to handle long-horizon tasks. Such a paradigm cannot actively reason about complex user intentions that often imply sequential affordances with multiple objects involved. In this paper, we introduce the Sequential 3D Affordance Reasoning task, which extends the traditional paradigm by reasoning from cumbersome user intentions and then decomposing them into a sequence of segmentations. Toward this, we construct the first instruction-based affordance segmentation benchmark that includes reasoning over both single and sequential affordances, comprising 180K instruction-point cloud pairs. Based on the benchmark, we propose our model, SeqAfford, to unlock the 3D multi-modal large language model with additional affordance segmentation abilities, which ensures reasoning with world knowledge and fine-grained affordance grounding in a cohesive framework. We further introduce a multi-granular language-point integration module to endow 3D dense prediction. Extensive experimental evaluations show that our model excels over well-established methods and exhibits open-world generalization with sequential reasoning abilities.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Hanqing_Wang4 ~Chunlin_Yu1 ~Haoyang_Luo2 ~Jingyi_Yu5 ~Ye_Shi1 ~Jingya_Wang3
Hanqing Wang Chunlin Yu Haoyang Luo Jingyi Yu Ye Shi Jingya Wang
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Multimodal Large Language Model 3D Affordance Reasoning
Yes, at the sentence level (e.g., fixing grammar, re-wording sentences)
I certify that there is no acknowledgement section in this submission for double blind review.
wang|seqafford_sequential_3d_affordance_reasoning_via_multimodal_large_language_model
/pdf/7f24f3cdda3fe3742cd6aee465949c6af0c8b3d1.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
No
Yes
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
SeqAfford: Sequential 3D affordance reasoning via Multimodal Large Language Model
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission