Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraints
Unsupervised learning;Interpretable representation learning;Emergent knowledge;Self-supervised learning
Yes
No ethics review needed.
- How does the choice of codebook size (K) affect interpretability and disentanglement performance?  

- The method uses specific values for the relativity parameter r and the V3 loss weight beta  depending on the dataset. Could you clarify the intuition behind selecting these values? How sensitive the performance is to different values of these hyperparameters?

- The paper mainly utilizes small datasets with specific content-style boundaries. How does it scale with larger datasets?

Minor issues: 
Line 463: The term "contennt" should be corrected to "content."
++ Unsupervised approach for separating style and content, which is a plus for domains where labeled data is unavailable or scarce

++ Generalizes across audio, image and video data. 

++ Content representations show interpretable properties. 

++ Performs well for out of distribution styles
They propose an unsupervised method to disentangle content and style representations.  They use vector-quantized autoencoder architecture and incorporate variance-versus-invariance constraints. Their method benefits from meta-level inductive bias, and shows out-of-distribution generalization. They demonstrate their method’s effectiveness across diverse applications such audio, images and video clips. Moreover, the learned codebook enables interpretable features.
-- They assume content elements are distinct, which can limit its application in domains with overlapping content.

-- The method is optimized for data with predefined fragments, which may not generalize well to continuous or unsegmented data types.

-- The experiments are done on relatively small datasets.
Yes
No ethics review needed.
1. It is better to evaluate V3 on  more complex or challenging datasets, such as face datasets FFHQ? And It is better to add discussion on 
 potential limitations of V3 when dealing with more complex data containing multiple contents or styles within a single sample.
2. Since V3 uses different losses to control content and style representations through self-supervised means, is there a risk that the representations intended to learn style might inadvertently capture content instead? How can it be ensured that there is not too much coupling between content and style representations? I suggest to conduct an analysis showing how much content information can be recovered from the style representations and vice versa.
3. Why is it necessary to apply VQ to content representations to form a codebook? What would be the impact on the learning results if VQ were not used? Seeing corresponding results in the Ablation study could be more convincing. 
4. Are the numbers of content and style representations in V3 the same? Considering that the actual samples may have varying quantities suitable for representing content and style, would setting different numbers for content and style representations in the same training round make a significant difference in the results?
1. The V3 method offers a novel unsupervised disentanglement approach that leverages the universal statistical differences between content and style. This is a creative application of variance-invariance constraints within an encoder-decoder framework, distinct from traditional methods that rely on supervision or paired data.
2. The loss calculation method proposed by V3, based on content and style, is concise and clear, aligning with human cognitive characteristics. Humans also tend to keep the style of the same sample similar while focusing on the content differences across different samples.
3. The model proposed in the paper is applicable to the learning of multiple modalities of style and content, including images, music, and animations, demonstrating strong generalizability. Moreover, the loss calculation pattern presented is not limited to the paper model but can also be applied to other related models for style and content learning.
4. The paper is well-structured, with a clear problem statement, methodology, and experimental evaluation. The use of datasets from multiple domains enhances the robustness of the research findings. The logic is strong, and the hierarchy is clear. The visualization of learned representations also helps readers better understand the results.
This paper introduces an unsupervised method named V3 (variance-versus-invariance) for disentangling content and style representations from a sequence of observations. The method is based on the assumption that content varies significantly within a sample but maintains a consistent vocabulary across samples, while style remains relatively invariant within a sample but varies significantly across samples. V3 integrates this inductive bias into an encoder-decoder architecture and demonstrates its effectiveness across multiple domains and modalities, including music audio, handwritten digit images, and simple animations. The paper reports that V3 outperforms existing unsupervised methods and even surpasses supervised methods in out-of-distribution generalization and few-shot learning. Additionally, V3 exhibits symbolic-level interpretability, aligning machine representations closely with human knowledge.
1. The paper's approach to learning sample content and style is reliant on datasets with a specific structure, which may limit its effectiveness when applied to images with varied or complex styles. For example, the Written Phone Numbers dataset contains only one digit in a specific position on each image, but the style of the digit can vary; in SPRITES, the character's position is fixed, and the form is also relatively uniform. If an image contains more content or more complex content, the model may have difficulties learning content and style.
2. The Ablation study reveals that certain components of the loss function may adversely affect performance metrics, suggesting a need for a deeper investigation into the roles and interactions of these components within V3.
3. It appears that V3 may have difficulty distinguishing between some similar contents during recognition. Similarly, although the paper may not mention it, V3 may also have similar problems with distinguishing styles, thus limiting the style discrimination between different samples.
4.Some important related works[a,b] are missing, which are also rely one some assumption to disentangle content and style.   The detailed comparison should be included, especially on the basic assumptions and the scope of suitable cases. 
a. Retriever: Learning Content-Style Representation as a Token-Level Bipartite Graph
b. Rethinking Content and Style: Exploring Bias for Unsupervised Disentanglement.
Yes
No ethics review needed.
See above.
1. The proposed method does not rely on external supervision such as labels.
2. The proposed method can generalize to unseen data.
3. The disentangled content representations align with human perceptions in some cases.
The paper proposes an unsupervised method that learns from raw observation and disentangles its latent space into content and style representations based on the meta-level insight of domain-general statistical differences between content and style.
1. The paper does not explicitly discuss how the codebook size and codebook dimension are determined. This could limit the application of the proposed method to different datasets.
2. The experiments are all conducted on toy datasets. This weakens the method's practical use in real-world scenarios. For example, for audio disentanglement, the musical note disentanglement is too simple. It is recommended to perform real-world disentanglement experiments such as voice conversion using VCTK.
Yes
No ethics review needed.
Can the author elaborate a bit more on the experiment setting of Section 4.2?
'we evaluate the models’ content-style disentanglement ability by conducting a retrieval experiment to examine the nearest neighbors of every input zc and zs using ground truth content and style labels, evaluated by the area under the precision-recall curve (PR-AUC) and the best F1 score'
1. This paper proposes an innovative disentanglement algorithm that does not rely on external labels of the style or content, making a novel contribution over the existing work, most of which requires external labels.

2. There is a clear rationale behind the proposed algorithm, and the rationale is interesting

3. The presentation in the paper is very clear.
This paper proposes an unsupervised disentanglement algorithm between styles and content, based on the assumption that style information remains consistent within samples, but varies across samples; whereas content information varies within samples, but the aggregated content is stable across samples due to the fixed vocabulary.
1. Although the proposed method does not rely on any external labels, it seems to be built upon rather strong assumptions. In particular, the method assumes that the aggregated content of a sample should stay constant across different samples, at least more stable than the style. This assumption is very strong, and there are many scenarios where this may not hold. First, if the content information in each example is too small to exhaust all the possible combinations, then the content would still vary greatly across samples. For example, in the written digit case, if each example only contains one digit. Then the model may no longer be very successful. This limitation may also hold for many other scenarios, such as Chinese calligraphy, which typically only contains very few characters, paintings containing only one object, etc. Second, if the content information exhibits too much variability, then the content across different statements may still have more variability than style. For example, consider short passages of different languages, where the content is the meaning of the passage and the style is language. The content in this case still varies greatly across different passages. Meanwhile, the authors also acknowledge that the method cannot deal with large content vocabularies. This limitation, together with the limitation discussed above, significantly limits the applicability of the proposed method.

2. Related to the first limitation, the experiment section seems to focus on very simple, carefully crafted toy scenarios that happen to satisfy the assumptions of the proposed method. There is no experiment on real-world scenarios, such as disentangling real human speech, photos, paintings, music, passages, etc. The lack of real-world experiments adds to the concern that the assumptions made by the proposed method may deviate too much from real-world cases to render the method useful.

3. (Minor) On page 5, the authors state that 'Theoretically, we aim to measure the consistency of codebook usage distribution along the sample axis, which is not differentiable'. This is not necessarily true. We can derive a soft, differentiable distribution of codebook usage via a softmax codebook selection. That is, for each codebook selection, rather than have a one-hot vector indicating which code is selected, we can derive a softmax selection, which is very common in deep-learning-based vector quantization methods, such as VQ-VAE. As a result, I believe that using the mean content vector to represent the distribution information may be improved.

4. (Minor) In line 242, the authors state that 'Content should be more variable within samples than across samples'. This is an overly simple statement. A more precise version should be 'Content should be more variable within samples than the sample-wise aggregated content across samples'.
Yes
No ethics review needed.
none
The paper is well written and easy to follow. The method is well motivated and described and results are nicely presented.
The authors propose a method to learn representations from an adapted VQ-VAE that disentangle content and style based on their different statistical characteristics: style is more constant over patches of a sample, whereas content can vary.
I have few comments on the main body of the paper, so please do not see this as a "few line review", I just do not have much to say.

**Experiments**: 
* 3 of 4 are on synthetic data with only SVHN being a "natural" data set and even that is cropped to the task. 
* It is of course right to demonstrate that the model performs well on data that follows its assumptions. 
* However, it would be informative (a) to see results on more natural datasets, and (b) to see how the model performs where assumptions do not hold so well, if only as a benchmark for future improvement.
We propose to use a meta-level inductive bias about content and style to learn their disentanglement, showing good out-of-distribution performance and enabling the emergence of interpretable content symbols that align well with human knowledge.
@inproceedings{
anonymous2024unsupervised,
title={Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraints},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=Lut5t3qElA},
note={under review}
}
We contribute an unsupervised method that effectively learns disentangled content and style representations from sequences of observations. Unlike most disentanglement algorithms that rely on domain-specific labels or knowledge, our method is based on the insight of domain-general statistical differences between content and style --- content varies more among different fragments within a sample but maintains an invariant vocabulary across data samples, whereas style remains relatively invariant within a sample but exhibits more significant variation across different samples. We integrate such inductive bias into an encoder-decoder architecture and name our method after V3 (variance-versus-invariance). Experimental results show that V3 generalizes across multiple domains and modalities, successfully learning disentangled content and style representations, such as pitch and timbre from music audio, digit and color from images of hand-written digits, and action and character appearance from simple animations. V3 demonstrates strong disentanglement performance compared to existing unsupervised methods, along with superior out-of-distribution generalization and few-shot learning capabilities compared to supervised counterparts. Lastly, symbolic-level interpretability emerges in the learned content codebook, forging a near one-to-one alignment between machine representation and human knowledge.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Unsupervised learning Interpretable representation learning Emergent knowledge Self-supervised learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/7a264514387a112cdb35194031b95251d71a9d40.pdf
unsupervised, self-supervised, semi-supervised, and supervised representation learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraints
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission