HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts
Generalized Category Discovery
Yes
No ethics review needed.
See weakness.
1. The paper presents a new, practically meaningful, and challenging setting, and constructs corresponding datasets.
2. The domain-semantic disentangled design is well-reasoned, clearly aligning the motivation.
3. The proposed approach demonstrates significant performance improvement on SSB-C.
4. The writing is clear and easy to follow.
This paper introduces a new challenge for Generalized Category Discovery, which requires model to categorize unlabeled data in the presence of domain shifts. Traditional GCD methods assume all images come from the same domain, which leads to a significant performance drop under domain shifts. The proposed HiLo framework explicitly disentangles semantics and domain, achieving domain adaptation in GCD through patchmix and curriculum learning. Experimental results show performance improvements, validating the effectiveness of the approach.
1. The performance gain on DomainNet is considerably smaller than on SSB-C, and the improvement over methods like SimGCD, which does not account for domain shift, is modest. This indicates limited robustness on various domain shifts and fails to highlight the advantages of the proposed approach.
2. The method is sensitive to certain hyperparameters, and $r^{'}$ does not exhibit consistent performance across the original and new domains.
3. The approach of decoupling domain and semantics is derived from [1], and the use of patchmix for domain adaptation is adapted from [2]. The curriculum learning strategy is also straightforward. Overall, the method seems to be an assembly of prior works, lacking substantial novelty.
4. There is no analysis of the disentangled domain and semantic features, such as distribution visualizations. This would help illustrate the effectiveness of the disentanglement.
5. In line 287, same representation loss $L^{rep}_s$ on both domain and semantic features is confusing. This approach may lead domain features to capture information beyond true domain characteristics. It would be valuable to see t-SNE visualizations of domain features, semantic features, and their combination. The author does not provide a corresponding discussion.
6. Line 313 mentions using pre-trained DINO to obtain $z_d$, but previously $z_d$ is associated with a projection head. If the projection head is discarded, then $z_d$ will always be identical in different time steps. If it is retained, the term “pretrained” is confusing. This needs clarification.
7. The ablation study is somewhat unclear. For instance, in row (5) where only deep features are used, does this mean all other designs related to the shallow feature $z_d$ are also omitted? This also needs clarification.

Reference

[1] Learning deep representations by mutual information estimation and maximization

[2] Patch-mix transformer for unsupervised domain adaptation: A game perspective
Yes
No ethics review needed.
Please see the Weaknesses.
1. The HiLo architecture extracts features from different layers of the vision Transformer and decouples domain and semantic features by minimizing mutual information. This feature processing method, based on the neural network hierarchical structure and information theory, provides a more effective feature representation for category discovery in the presence of domain shifts and avoids the problem of feature confusion in traditional methods.

2. The PatchMix method is introduced into the GCD task and innovatively extended. By adjusting its objective function, it can adaptively utilize labeled and unlabeled data for training. This extension not only combines the advantages of data augmentation but also flexibly adjusts the learning process according to the nature of different data, enhancing the model's ability to learn data from different domains and categories.

3. The curriculum learning method is employed, which dynamically adjusts the sampling probability weights according to the difficulty of samples and the unknown degree of domains. This strategy of gradually introducing samples from easy to difficult conforms to the learning law, enabling the model to better adapt to the challenges brought by domain shifts and improving the model's convergence speed and robustness to complex data distributions.

4. In terms of method design, innovative technical architectures and learning strategies are used, as well as theoretical analyses to verify their effectiveness. From the theoretical derivation of the target error to the analysis of the roles of different components, a solid theoretical foundation is provided for the innovation of the method, demonstrating the advantage of the close integration of theory and practice.
Generalized Category Discovery (GCD) is a challenging task where, given a partially labeled dataset, the model must classify all unlabeled instances. This paper introduces a new task and method to handle the GCD problem when the unlabeled data contains images from different domains. In terms of the method, the HiLo architecture and learning framework involves extracting "low-level" (early layers) and "high-level" (late layers) features from a vision Transformer and decoupling domain and semantic features by minimizing the mutual information between the two sets of features. The PatchMix contrastive learning method is introduced into the GCD task, with its objective function extended to enable the utilization of both labeled and unlabeled data for training. Curriculum learning is adopted, gradually increasing the sampling probability weight of samples predicted to be from unknown domains to enhance the model's robustness to domain shifts. Experiments are conducted on the DomainNet and the SSB-C benchmark datasets constructed based on the Semantic Shift Benchmark (SSB). The experimental results show that HiLo significantly outperforms existing category discovery models, validating the effectiveness of the method.
1. In HiLo, features are disentangled by assuming that features from different layers represent domain and semantic information, respectively and minimizing the mutual information based on this assumption. However, this assumption may oversimplify the complexity of feature representation in neural networks. In fact, features from different layers may be a mixture of multiple types of information. Simply defining the early layers as domain features and the late layers as semantic features may not be entirely accurate, which may lead to incomplete feature disentanglement in some complex data distributions and affect the performance and generalization ability of the model.

2. The introduction and extension of PatchMix in the GCD task is an innovation, but it also brings some problems. The adjustment of its objective function and its application on different data increases the complexity of the model. When dealing with data with large domain differences, it is a challenge to determine the mixing proportion and application method accurately. If not handled properly, it may introduce too much noise or incorrect information, which may instead interfere with the learning process of the model and reduce the classification performance.

3. In the curriculum learning method, the adjustment parameters of the sampling probability weights need to be selected through the validation set, which increases the dependence of the model on specific datasets. Moreover, for different datasets and tasks, the optimal values of these parameters may vary greatly, and the model cannot adaptively determine these parameters. If these parameters cannot be correctly selected in a new dataset or task, curriculum learning may not be able to play its intended role. It may even have a negative impact on the learning of the model.
Yes
No ethics review needed.
1. For the experimental results of the ORCA method, what is the backbone used by the authors?
2. Were any curriculum learning alternatives considered, such as adaptive weighting based on difficulty or dynamic sample weighting? A brief discussion on these choices would clarify why the current approach was favored.
1. The paper presents an innovative GCD approach by combining mutual information minimization with domain-specific data augmentation and curriculum learning to handle domain shifts effectively.
2. Extensive evaluation on both synthetic (SSB-C) and real-world (DomainNet) benchmarks demonstrates the model's robustness and its superiority over baseline GCD models, especially under domain-shifted conditions.
The paper introduces the HiLo framework, a learning method aimed at tackling Generalized Category Discovery (GCD) under domain shifts. HiLo addresses challenges in categorizing both seen and unseen categories across distinct domains within partially labeled datasets, leveraging a multi-faceted approach: mutual information minimization to separate domain and semantic features, PatchMix for augmented domain adaptation, and a curriculum learning strategy. The proposed method is evaluated on synthetic and real-world domain-shift datasets, showing substantial improvements over existing GCD models.
1. In the "Problem statement," the following sentence is unclear: "The objective of GCD is ... with singleton cardinalities for the latter." The author needs to differentiate between the GCD task setting and the domain shift GCD task setting, so this statement should be revised for clarity and precision.
2. The font sizes of the tables are not standardized, and the font in table 2 is too small.
3. I'm curious as to how many runs each of the authors' experimental results were derived from, and given that the differences in the results of the GCD benchmark tests can be very large, the authors should have listed the error lines generated by the three independent runs.
Yes
None
No ethics review needed.
Please clarify the novelty of the proposed method, and include more comparisons with UinOT in the main results.
The paper proposes a new problem setting and proposes a HiLo, which combines multiple techniques from domain adaption and achieves better results.
This paper introduces a new problem setting: Generalized Category Discovery (GCD) with domain shift. The authors leverage techniques from domain adaptation and curriculum learning to propose a new method called HiLo. Comprehensive experiments on the proposed benchmark demonstrate substantial improvements.
1. The novelty of the method appears limited, as it seems to combine various techniques from different domains.

2. The comparison with UniOT should be included in the main results. Since the proposed setting is similar to universal domain adaptation, it is essential to compare methods from both domains in the main results.

Minor： 

Missing citation for the following important paper

[1] Rastegar et al. Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery. NeurIPS 2023.

[2] Gu et al. Class-relation Knowledge Distillation for Novel Class Discovery. ICCV2023.
we extracts domain and semantic information independently and minimize their mutual information while incorporating contrastive learning for robust representations with pseudo-labelling strategies
@inproceedings{
anonymous2024hilo,
title={HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=2eFq6S35iB},
note={under review}
}
Generalized Category Discovery (GCD) is a challenging task in which, given a partially labelled dataset, models must categorize all unlabelled instances, regardless of whether they come from labelled categories or from new ones. In this paper, we challenge a remaining assumption in this task: that all images share the same \underline{domain}. Specifically, we introduce a new task and method to handle GCD when the unlabelled data also contains images from different domains to the labelled set. Our proposed `HiLo' networks extract High-level semantic and Low-level domain features, before minimizing the mutual information between the representations. Our intuition is that the clusterings based on domain information and semantic information should be independent. We further extend our method with a specialized domain augmentation tailored for the GCD task, as well as a curriculum learning approach. Finally, we construct a benchmark from corrupted fine-grained datasets as well as a large-scale evaluation on DomainNet with real-world domain shifts, reimplementing a number of GCD baselines in this setting. We demonstrate that HiLo outperforms SoTA category discovery models by a large margin on all evaluations.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Generalized Category Discovery
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/c6c6274e20c9ed743ebf95ea90ca9e108a82e0b3.pdf
unsupervised, self-supervised, semi-supervised, and supervised representation learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission