Gaussian-Based Instance-Adaptive Intensity Modeling for Point-Supervised Facial Expression Spotting
micro-expression spotting;semi-supervised learning;soft pseudo-labeling
Yes
No ethics review needed.
Please refer the the questions.
1. The setting of point-supervised facial expression spotting is more practical than fully-supervised facial expression spotting.

2. The proposed intensity-aware contrastive learning is interesting. Rather than simply pulling together the same-class instances and pushing apart the different-class instances, the authors consider the influences of intensity, i.e., less attention should be paid to the same-class instances with large intensity differences, and to the different-class instances with small intensity differences.
This paper treats facial expression spotting as a point-supervised temporal action localization (P-TAL) problem. Drawing from existing P-TAL techniques, the authors propose a two-branch framework for localizing facial expressions. One branch, known as the action-classification branch, predicts the category scores of the facial expression $S$, while the other branch predicts its intensity score $a$.

The framework utilizes a proposed Gaussian-based Instance-Adaptive Intensity Modeling approach to establish soft pseudo-labels for each frame based on the predicted intensity and the given point label. The expression intensity branch is optimized by minimizing the difference between the predicted intensity and the soft pseudo-label, while the action-classification branch employs cross-entropy loss, with class labels also derived from the soft pseudo-label. To reduce neutral noise further, the authors introduce an intensity-aware contrastive loss.

The proposed method is evaluated on two datasets containing both macro and micro facial expressions. Experimental results indicate that this approach surpasses state-of-the-art P-TAL methods in the task of facial expression spotting.
1. In the proposed method, the intensity score is shared across all facial expression classes. However, two or more facial expressions may occur within a short clip, each with distinct apexes. This raises a concern: what happens if the method is applied to an untrimmed video containing two closely timed apexes of different facial expressions? It would be beneficial to discuss how the model handles this scenario.

2. The presentation of the inference lacks sufficient detail. Specifically, it would be helpful to clarify what the outer-inner-contrastive score is and how these scores contribute to determining the temporal range and position of the facial expressions.

3.  The soft pseudo-label is established based on the predicted intensity scores. However, these predicted intensity scores can fluctuate throughout the training process. It would be helpful to clarify how the soft pseudo-label is updated in response to these changing intensity scores.

4. The legend of Fig 4 is unclear. The marks of ground-truth boundary frame are missing.
Yes
N/A
No ethics review needed.
Please address the concerns in the Weaknesses section.
- The paper addresses interesting issue of facial expression spotting.
- Ablation study has shown some contributions of the proposed terms.
In this paper, the author introduced a two-branch framework for Point-supervised facial expression spotting (P-FES) task. 
The proposed framework consists of two main branches: (1) Expression Intensity branch with a proposed Gaussian-based instance-adaptive Intensity Modeling (GIM) to model the intensity distribution of expression proposals; and (2) Action-Classification branch to classify the expression class from the proposals.
An Intensity-Aware Contrastive Learning on pseudo-labeled frame is also presented to enhance the discriminative feature learning process.
Experiments on SAMM-LV and CAS(ME)2 have shown the advantages of the proposed approach.
Although the soft-label is well-motivated, it has been widely used as a label smoothing technique in literature.
Moreover, the training process is complicated and ad-hoc with 3 training stages. However, the improvement is minor and cannot out-perform F-FES techniques.


1. During the training process, warm-up epochs are very important as both features and scores are not reliable for mining steps (i.e. extract pseudo-apex frame, gaussian modeling and reliable pseudo-neutral frames). Then, hard label technique is required. As a result, the proposed framework can only give a minor improvement on top of hard label technique. 

2. In the proposed framework, one of the mining step is to extract pseudo-neutral frames. This will be challenging for ME classes as the micro-expression frames will be very similar to neutral frames and their feature difference is minor. How can the proposed framework extract reliable pseudo-neutral frames and achieve better performance on ME classes ?

3. How can the duration of each expression class be chosen?

4. More analysis on the pseudo-apex frame is recommended. 
For example, How are the pseudo-apex frames changing during different training epochs?
Yes
No ethics review needed.
1、Not all facial expression instances conform to a Gaussian distribution in every situation, such as when someone is suddenly startled. The authors do not seem to address how to handle this special case.

2、I hope that Figure 4 can provide a clearer explanation.
1、The motivation of this article is well-founded, and the proposed method effectively addresses the problem of point-supervised facial expression recognition.

2、The experiments are quite comprehensive, and the comparisons with other methods are reasonable.
The article proposes a two-branch framework for Point-supervised Facial Expression Spotting (P-FES), which includes a Gaussian-based Instance-Adaptive Intensity Modeling (GIM) module for soft pseudo-labeling. This framework aims to address the problem of locating facial expression instances in untrimmed videos, requiring only a timestamp label for each instance during training.
Some details are unclear; see the "Questions" section below for more information.
An instance-adaptive Gaussian-based soft pseudo-labeling method for point-supervised facial expression spotting, which models the expression intensity distribution at the instance level.
@inproceedings{
anonymous2024gaussianbased,
title={Gaussian-Based Instance-Adaptive Intensity Modeling for Point-Supervised Facial Expression Spotting},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=daD6uGMeLs},
note={under review}
}
Point-supervised facial expression spotting (P-FES) aims to localize facial expression instances in untrimmed videos, requiring only a single timestamp label for each instance during training. To address label sparsity, hard pseudo-labeling is often employed to propagate point labels to unlabeled frames; however, this approach can lead to confusion when distinguishing between neutral and expression frames with various intensities, which can negatively impact model performance. In this paper, we propose a two-branch framework for P-FES that incorporates a Gaussian-based instance-adaptive Intensity Modeling (GIM) module for soft pseudo-labeling. GIM models the expression intensity distribution for each instance. Specifically, we detect the pseudo-apex frame around each point label, estimate the duration, and construct a Gaussian distribution for each expression instance. We then assign soft pseudo-labels to pseudo-expression frames as intensity values based on the Gaussian distribution. Additionally, we introduce an Intensity-Aware Contrastive (IAC) loss to enhance discriminative feature learning and suppress neutral noise by contrasting neutral frames with expression frames of various intensities. Extensive experiments on the SAMM and CAS(ME)$^2$ datasets demonstrate the effectiveness of our proposed framework.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
micro-expression spotting semi-supervised learning soft pseudo-labeling
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ed09c599378aa3a9357dc108a6e87e1570090ff0.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Gaussian-Based Instance-Adaptive Intensity Modeling for Point-Supervised Facial Expression Spotting
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission