DiffSign: A Different Approach to Continuous Sign Language Recognition with Diffusion Model
Continuous sign language recognition;Video-to-text generation;Weakly supervised;Contrastive learning
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
Completing continuous sign language recognition through optimized video-to-text generation.
@misc{
geng2024diffsign,
title={DiffSign: A Different Approach to Continuous Sign Language Recognition with Diffusion Model},
author={Xi Geng and Yunan Li and Zhuoqi Ma and Qiguang Miao},
year={2024},
url={https://openreview.net/forum?id=GPfEMmgspR}
}
Cross-modal alignment is a general way for continuous sign language recognition (CSLR) tasks. However, Due to the weakly supervised nature of CSLR, manual alignment often fails to map sign frames to glosses accurately. 
In this paper, we propose a diffusion-based framework, achieving CSLR in a new view based on cross-modal generation, leveraging the inherent semantic consistency between sign videos and glosses. To address the issue of ambiguous boundaries in sign videos, we have also developed a contrastive learning-based feature enhancement strategy, which serves as a more sophisticated alternative to the simple attention mechanisms commonly used in text-to-image generation tasks. 
Extensive experiments on three public sign language recognition datasets demonstrate the effectiveness of generation way in CSLR and it can achieve better performance than state-of-the-art methods.
The code of our method will be available upon acceptance.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Xi_Geng2 ~Yunan_Li1 ~Zhuoqi_Ma1 ~Qiguang_Miao1
Xi Geng Yunan Li Zhuoqi Ma Qiguang Miao
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Continuous sign language recognition Video-to-text generation Weakly supervised Contrastive learning
I certify that there is no acknowledgement section in this submission for double blind review.
geng|diffsign_a_different_approach_to_continuous_sign_language_recognition_with_diffusion_model
/pdf/4d3a30efba0ca33aa3cdc0799af45465a7029044.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/9d8c2232c8aec2fa56d004b06aee4f4dd9fc8e8e.pdf
DiffSign: A Different Approach to Continuous Sign Language Recognition with Diffusion Model
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission