Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions
unsupervised learning;identifiability;unknown latent dimension
Yes
I have no concerns.
No ethics review needed.
- Do you believe that L1​ regularization provides a strong enough sparsity constraint to justify the Domain Variability assumption? Could you elaborate on why you chose L1 ​over potentially stricter sparsity-inducing methods, particularly given its central role in ensuring the injective property of the mapping function.
- The model relies on some broad assumptions. Could you discuss the practical conditions that need to be met to ensure invertibility, and how sensitive the method might be to deviations from these conditions?
- Have you observed scenarios where the L1 constraint may not be strong enough to prevent information leakage between content and style representations?
- It appears from the results that the content representation often corresponds to pose, while the style representation captures semantic attributes. Is this always the case, or have you observed instances where content and style encode different aspects of the data? For example, in Figure 3(b), it seems that changing the domain also changes hair color.
- It would be helpful to include experiments on simpler datasets, such as rotated MNIST, to provide a clearer demonstration of the model’s ability to separate content and style. For example, with rotated MNIST, the content could represent class information, while rotation and other stylistic aspects could be designated as style. Such examples could offer readers a more intuitive understanding of how content and style representations are disentangled and controlled within the model.
- The paper is well-structured and clearly written.
- It introduces a unique and interesting distribution matching perspective for content and style transfer in image translation
- By relaxing restrictive assumptions often required in previous work—such as the need for component-wise independence and pre-defined latent dimensions—the approach enhances practicality and scalability.
- The paper provides both a theoretical foundation for content-style identifiability and promising qualitative results.
This paper enhances the identifiability of latent content and style representations in unaligned multi-domain data by framing the problem within a distribution matching framework. The authors establish that, under relaxed conditions, their approach does not require restrictive assumptions like component-wise independence or known latent dimensions. By applying sparsity constraints, they ensure identifiable content-style separation, and they ultimately reformulate the distribution matching objective into a simpler GAN training framework. The authors empirically demonstrate improvement in image translation.
- The paper’s dimension-agnostic approach relies heavily on sparsity constraints to prevent information leakage between content and style representations, which may not be effective in all cases.
- The paper relies on several broad assumptions, such as the invertibility of the function up to certain ambiguities, without fully detailing the specific conditions under which this assumption holds.
- The L1 regularization appears to me to be too "soft" of a sparsity constraint to adequately justify the injective property of the mapping function $f$, particularly given that the model relies heavily on this constraint to ensure generative model is a valid invertible function.
- The paper, in general, lacks a thorough ablation study to validate the author's claims.

$\textbf{Minor Remarks}$
- Line 741, I think $\mathcal{A}\in\mathcal{Y}$ rather than $\mathcal{A}\in\mathcal{X}$.
- This is a minor suggestion, but it may be worth considering alternative abbreviations for "DM" and "LDM" due to the common use of these terms in diffusion models and latent diffusion models.
Yes
No ethics review needed.
1. In Theorem 4.1,  $q$ is a bijective and $f=q^{-1}$, thus trivially $f$ becomes an bijective, which somehow is inconsistent with the fact    that $f$ is a injective from $\mathcal{X}$ to $\mathbb{R}^{\hat{d}_{c}+\hat{d}_s}$ , 
   can the authors clarification? 

   Particularly on the relation between $\mathcal{C}$, $\mathcal{S}$ and $\mathbb{R}^{\hat{d}_{c}+\hat{d}_s}$ as they are actually different spaces. 
2. In Assumption 3.1, $N$ is the dimension of style variables, but somehow in later part of the paper it has different meaning, can the authors clarify? Similarly, the notation $n$ also has different meanings in different part of the paper.
3. Can you provide the correct citation for Proposition E.1?
1. The writing of the paper is good. Most parts of the paper are easy to follow and understand. 
2. The experimental results are good.
The authors try to attach the problem of identifying latent content and style variables from observations from un-aligned domains. The authors claim their main contribution as follows: (1) t restrictive assumptions such as component-wise independence of the latent variables can be removed; (2)  knowledge of the content and style dimensions is not necessary for ensuring identifiability. However, there are some inconsistencies between the main results and Theorem 4.1.
There may be some inconsistency between  the main results and Theorem 4.1. In the main results, it claims that $f$, $\gamma$, and $\delta$ are injective. However, in theorem, the $q$ is a bijective and $f=q^-1$, which implies that $f$ will be an bijective. As a result, $\gamma$ and $\delta$ becomes bijective and it implies the dimension of content and style is actually $\hat{d}_c$ and $\hat{d}_s$. This causes a contradiction in the theory.

The proof of Theorem 4.1 uses  Proposition E.1 from  (Zimmermann et al., 2021) https://arxiv.org/pdf/2102.08850. However, I did not find the Proposition in the reference. Can the authors fix the problem?
Yes
No ethics review needed.
Can the authors please add some remarks regarding the potential weaknesses mentioned above, regarding the suitability of their approach for text, and regarding the potential instability of adversarial GAN training used in their approach?
- The paper provides a good overview of background and related work on multi-domain content-style modeling, content-style identifiability, and the challenge of knowing the content and style dimensions apriori. 
- The contributions in this paper appear to be novel and technically sound from both a theoretical and experimental standpoint.
- The proposed approach in this paper appears to be reproducible, given the provided source code and experimental details in the paper.
- The experimental results show that the proposed approach performs well on multi-domain image translation and generation tasks.
This paper tackles the problem of identifiability of latent content and style variables from unaligned multi-domain data. A new approach based on cross-domain latent distribution matching (LDM) is presented, which removes restrictive assumptions from prior work regarding component-wise independence of latent variables and knowledge of content and style dimensions. The LDM formulation is implemented as a regularized multi-domain GAN loss with coupled latent variables. In addition to theoretical contributions on these topics, the authors present experimental results on image translation and generation tasks.
- Because the proposed approach is implemented as a GAN, it seems likely that this approach will be most suitable for images (and possibly audio), and may be problematic for discrete data modalities, particularly text. 
- The authors don’t directly mention or address potential instabilities that may result from adversarial training of their GAN-based approach.
Yes
No ethics review needed.
Please see the weaknesses part.
Strengths:
- This paper is well-written, well-motivated, and easy-to-follow. Both the experiments and theoretical justification are intuitive and convincing.
- The contribution of easing independent assumption is helpful for developing simple and efficient causal models.
- A sparsity-regularized implementation is proposed to solve cross-modal learning efficiently.
Summary:
This paper studies causal identification of latent factors content and style in the cross-domain learning problems. Specifically, existing works assume that all latent components are independent to each other. Such an assumption is strict and could not be fulfilled in realistic scenarios. In this paper, the authors propose a cross-domain latent distribution matching framework which does not require strict independence between different latents. Moreover, the proposed framework can ensure content style identification can be free from the specific latent dimensions which further ease the assumption for tackling the problem. Through both theoretical and empirical justifications, the authors carefully testified the effectiveness of the proposed method.
Weaknesses:
- The proposed target in Equation 4 aims to minimize the norm of style latents by enforcing distribution matching across content and block-independence between content and style. However, it could lead to a degenerate solution when there is misidentification between content and style, and some content features already achieves small norm. Could you explain how can the proposed identifiability result can avoid the problem from happening?
- The computational efficiency of the framework is unknown. Only numerical result of generation is provided, there is no quantitative comparison to show the efficiency compared to other baseline methods. Moreover, why sparsity regularization is needed for training GAN needs a justification.
- Missing related references:  
Hong et al., Improving Non-Transferable Representation Learning by Harnessing Content and Style, in ICLR 2024.  
Huang et al., Harnessing Out-Of-Distribution Examples via Augmenting Content and Style, in ICLR 2023.  
Dai et al., Disentangling writer and character styles for handwriting generation, in CVPR 2023.
Yes
No ethics review needed.
1. Can you further explain the reasonability using Assumption 3.1? Can you relax it? If not, what are the difficulties of relaxing this assumption? Furthermore, in what real-world scenarios might this assumption be violated?
2. Can you explain the gap between your method in 4(a) and the way you do implementation? What is the motivation for not using (4) directly? Any trade-offs or limitations of their implementation approach compared to directly using formulation (4).
The paper is well written and the contribution is clear.
This paper introduces an analytical framework via cross-domain latent distribution matching(LDM), which establishes content-style identifiability under substantially more relaxed conditions.
The author seems to slip the main result part (3.2) too quickly (with no more than half a page). I am unsure what the key difference between your approach and other low-rank techniques is. In addition, there seems to exist a gap between the optimization problem (4a)(4b)(4c) and the implementation. (I mean: why not to use the formulation in implementation section as the theoretical modeling approach in the methodology part?) There are some writing issues in the paper too, e.g. you can not directly quote (4) , you should use 'according to formulation (4)'.(e.g. the writing in Theorem 4.1)
@inproceedings{
anonymous2024contentstyle,
title={Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=p60Y6o85Cj},
note={under review}
}
Understanding identifiability of latent content and style variables from unaligned multi-domain data is essential for tasks such as
domain translation and data generation. Existing works on content-style identification were often developed under somewhat stringent conditions, e.g., that all latent components are mutually independent and that the dimensions of the content and style variables are known. We introduce a new analytical framework via cross-domain *latent distribution matching* (LDM), which establishes content-style identifiability under substantially more relaxed conditions. Specifically, we show that restrictive assumptions such as component-wise independence of the latent variables can be removed. Most notably, we prove that prior knowledge of the content and style dimensions is not necessary for ensuring identifiability, if sparsity constraints are properly imposed onto the learned latent representations. Bypassing the knowledge of the exact latent dimension has been a longstanding aspiration in unsupervised representation learning---our analysis is the first to underpin its theoretical and practical viability. On the implementation side, we recast the LDM formulation into a regularized multi-domain GAN loss with coupled latent variables. We show that the reformulation is equivalent to LDM under mild conditions---yet requiring considerably less computational resource. Experiments corroborate with our theoretical claims.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
unsupervised learning identifiability unknown latent dimension
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/1d7a086f793cf3ba0d5ff2eacd5a8dad0d32e3d8.pdf
unsupervised, self-supervised, semi-supervised, and supervised representation learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/5fcf9dffcedaed5dc2b6817bc874be6b010224ca.zip
Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission