DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization
Unsupervised Learning;Shape Analysis;Identifiability in Representation Learning;Disentangled Representation Learning;ML in Biology
Yes
No ethics review needed.
Please refer to the Weaknesses section.
1. This paper proposes well-defined conditions for the disentanglement of content and transformation. 
2. The experiments are conducted on four datasets, and comprehensive qualitative results are shown.
This paper proposes an unsupervised disentangling method to disentangle content and transformation of the input. Specifically, this paper first proposes two conditions that disentanglement of content and transformation should satisfy. Then, this paper proposes a method to construct positive and negative samples with respect to both content and transformation. The key idea is to utilize a variational autoencoder to construct these samples. The experiments are conducted on four datasets, i.e., three of them (mnist, linemod, and starmen) are pure images, and one is protein subtomogram. One quantitative result and several qualitative results are shown to prove the effectiveness of the proposed method.
The main concern of this paper is evaluation, which is insufficient and less significant.
1. The first three datasets (mnist, linemod, starmen) are somehow toy datasets, which is less significant in real-world applications.
2. I agree that protein conformation is one meaningful real-world application, but other than map visualizations, it fails to produce convincing evaluation results.
3. There lacks some widely used evaluating metrics in Table 1 to demonstrate the application of the disentanglement.
4. This paper also does not provide comparisons with other baseline methods or state-of-the-are methods.
Yes
No ethics review needed.
In the method section, condition 1 is very confusing. I think it was meant to be "for all $T \in T$ and $x \in X, h_c(T(x)) = h_c(x)$", but please correct me if I misunderstood this.

In the method section, many terms are used seemingly interchangeably, such as "latent space", "factor", "representation", "transformation", "content". Please clarify these terms. For example, at line 206, "transformation" is used, however, I think it was meant to be "transformation representation" or "transformation factor", unless I misunderstood.

There are a few grammatical and syntactical mistakes, such as inconsistencies in the use of uppercase and lowercase, sometimes writing "shape focused" while other times "shape-focused".
Originality.
In this work, a novel method to address the problem of creating positive pairs of transformations under content change has been proposed. The core of this work is original.

Quality.
The method proposed was evaluated on a sufficient number of datasets. Although not very complex, they could suffice in showing the potential for this approach. The baselines chosen are also relevant to the method proposed.

Clarity.
The figures in the experiment section allow for a quick qualitative assessment of the performance of the methods. The method explanation is quite clear.

Significance.
This work and the proposed method have shown some potential for successful applications.
Unsupervised disentanglement of transformations and content is a challenging task that was previously approached primarily through using separate ad-hoc transformation methods, or by self-supervised contrastive-based methods. Ad-hoc transformations suffer from being limited to the given parameterization chosen, while self-supervised methods do not tackle this disentanglement problem directly. In this work, DualContrast is proposed, which consists of a VAE with additional contrastive losses designed to disentangle content and transformation. The hardest challenge is obtaining positive pairs of samples with respect to transformations: changing the content while keeping the transformation constant. In this work, this has been done by decoding two random samples from the prior of the transformation latent space while feeding different permutations of the content latent representation to obtain similar transformations with different content. The method is applied to MNIST, LineMod, Starmen Shapes, and Cryo-ET subtomograms with positive results.
The explanation of the method in the abstract and introduction is especially unclear. This is also a problem because Figure 2 fails to properly and intuitively show the method. Reading the method section explains this more. To improve, I would suggest clearly highlighting  the role of the latent space in the creation of the positive pair that would otherwise be impossible. This could be done similarly to how it was done in Figure 3. Figure 2 would then become useful. Additionally, Figure 2 lacks proper annotations such as labeling of all elements present, and proper caption explaining what happens in the figure in a more complete way. There is some inconsistency in how things are called. In the figure, style, and content are mentioned, however, in the text it is clear that "style" is supposed to be "transformation", please pick one and stick with it in the whole manuscript, either one would suffice, however, transformation is likely to be more accurate.

The contributions are a bit bold. The first contribution, especially, is more context for the work than a contribution and could be removed entirely. Please consider reworking the contributions to be more reflective of the actual content.

The related work section should expand a bit more on the protein part, which is currently very unclear for somebody who is not a practitioner. Please provide more examples, even referring to the appendix to understand the data and the context better.

The method section has a few mistakes and the explanation is very wordy, which makes it hard to follow. I wrote a few observations in the questions section of this review.

The manuscript should include the limitations of this method, especially regarding the latent space-based approach to creating positive transformation pairs. For example, the limitations should address whether this approach could be extended to real-world datasets or whether this approach should be limited to specific types of datasets. 

The experiments lack in quantitative results. Although disentanglement is very hard to measure, given the ability to choose datasets, it would be much more convincing to have datasets where a quantitative assessment is possible either in the form of direct supervision (similar to what the disentanglement metric is currently doing), or through some downstream tasks where the disentanglement would be useful. Such tasks could be segmentation, or visual question answer.
The choice of the "human deformation" as a dataset is very confusing, and the results reported are also very underwhelming. Although the generated shapes are better, the data appears to be very trivial, so some more information on the training and the difficulty of fitting such samples would be more convincing. The number of parameters, ablation performed, and results from more baselines would be a step in the right direction. It is especially important to keep including all baselines for all datasets used, the results on human deformation appear to be unfinished. If so, it would have been better to simply exclude the dataset from the manuscript.
Additionally, plots of the latent space are only available for the cellular dataset
Yes
n/a
No ethics review needed.
- On L147, the authors say that they model "is highly effective in disentangling wide range of transformations from the content in various shape-focused image datasets by only using simple rotation for creating contrastive pairs since the representation for disentangled rotation generalizes over other shape transformations." Could they elaborate on this? Why is it the case? Where is it shown on the paper? How can we be sure it will work to other modalities besides the toy tasks tested?
- Is the VAE trained at the smae time as the contrastive losses? Since the VAE is used to generated samples for the CLs, how do training jointly vs training in two stages (VAE followed by CLs) change the performance?
+ The paper is well written and easy to follow
+ The idea of disentangling features is an important problem in many applications of machine learning
+ The proposed approach is simple and well motivated
In this paper, the authors propose a model that learn unsupervised representation for  "shape-focused images". In particular, their method, DualContrast, learn to disentangle "content" and "transformation" in an unsupervised fashion. The model is trained with an a combination of 2 contrastive losses (one for context, one for transformations) and a VAE loss (the VAE is used to sample positive samples for the CL of transformations). The authors show results of the proposed model on multiple small/toysh datasets.
-  The author often mention that the work focus on "shape-focused real-world images", but they only applied in very simplified, toysh settings, very far from "real-world images". Even the CryoEM task is a very simplified task.
- The choice of positive/negative samples for each factor is very ad-hoc. The paper lacks explanation and empirical validation on why this choice makes sense versus others. 
- I found very strange the choice of using VAE generated samples as data to train the contrastive loss. This idea of using generated smples to train a model is not well understood. This approach might have worked in the very simplified tasks tested on the paper, but It is very unlikely that the proposed model would work on any real-world dataset.
- I also find the experimental results a bit weak. First, the datasets utilized in this work are very simple and results on them probably wont guarantee their utility on real-world problems. Second, the metrics utilized on Table 1 are not particularly significant. Third, most of the results are qualitative and based on one or two images that can potentially be cherry picked.
An unsupervised method for content-transformation representation disentanglement
@inproceedings{
anonymous2024dualcontrast,
title={DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=EMKZyZSl70},
note={under review}
}
Unsupervised disentanglement of content and transformation is significantly important for analyzing shape-focused real-world scientific image datasets, given their efficacy in solving downstream image-based shape-analysis tasks. The existing relevant works address the problem by explicitly parameterizing the transformation factors, significantly reducing their expressiveness. Moreover, they are not applicable in cases where transformations can not be readily parametrized. An alternative to such explicit approaches is contrastive methods with data augmentation, which implicitly disentangles transformations and content. However, the existing contrastive strategies are insufficient to this end. Therefore, we developed a novel contrastive method with generative modeling, DualContrast, specifically for unsupervised disentanglement of content and transformations in shape-focused image datasets. DualContrast works by creating positive and negative pairs for both content and transformation factors from data and latent spaces. Our extensive experiments showcase the superiority of DualContrast over existing self-supervised and explicit parameterization approaches. We leveraged DualContrast to disentangle protein identities and protein conformations in cellular 3D protein images. Moreover, we also disentangled transformations in MNIST, viewpoint in the Linemod, and human deformation in the Starmen dataset as transformations using DualContrast in an unsupervised manner.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Unsupervised Learning Shape Analysis Identifiability in Representation Learning Disentangled Representation Learning ML in Biology
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/dedf2b1fff4a19b402d78263618f0b98935c6cd7.pdf
applications to physical sciences (physics, chemistry, biology, etc.)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/2e9616f6c5428182f64219230d741c442a52fbc4.zip
DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission