FAACL: Federated Adaptive Asymmetric Clustered Learning
Federated Learning;Distributed Learning;Clustering
Yes
No ethics review needed.
Q1 :  line 053 - “and all existing techniques assume that each device contributes to a single cluster.” - What does this line mean here? And how do address the same in your paper? 

Q2: Does it have the same privacy guarantees as other models proposed in the same vein such as [1] and others. 

Q3: How does the computational complexity compare to other algorithms? 

Q4: How far off is this clustering from ground truth? Additionally is there a comparison again Oracle? [2] 

Q5: Was there an ablation study performed without merging of clusters and just having asymmetric contribution? 

[1] Sattler, Felix, Klaus-Robert Müller, and Wojciech Samek. "Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints." IEEE transactions on neural networks and learning systems 32.8 (2020): 3710-3722.

[2] Jothimurugesan, Ellango, et al. "Federated learning under distributed concept drift." International Conference on Artificial Intelligence and Statistics. PMLR, 2023.
S1: Introduced asymmetric clustering 
S2: Source code is shared for reproducibility.
Authors discuss the clustered learning approach in Federated Learning and relax the following constraints:
1. Number of clusters being fixed (Adaptive)
2. Each device contributes to a single cluster (Adaptive) 
3. Model sharing is reciprocal in nature (Asymmetric)
W1: This comment is in relation to the Section 4.2 and 4.3 -> Some of the related works may need to be studied and probably be included as part of the comparison metrics. [1-2]

W2: How does this handle a newly added device in the Federated Learning? Do we check against all the clusters and find if it supports or not individually or against the clusters only? Does that a performance hit if the clusters are already prepared.  

[1] Vardhan, Harsh, Avishek Ghosh, and Arya Mazumdar. "An Improved Federated Clustering Algorithm with Model-based Clustering." Transactions on Machine Learning Research (2024).

[2] Kim, Heasung, Hyeji Kim, and Gustavo De Veciana. "Clustered Federated Learning via Gradient-based Partitioning." Forty-first International Conference on Machine Learning.
Yes
No ethics review needed.
See weaknesses
1. The proposed approach tackles the underexplored issue of asymmetric clustering in Clustered Federated Learning, providing a fresh perspective.

2. The experimental results are comprehensive.
This paper proposes a new federated learning technique where some devices may contribute to the training of the models of other devices, but without enforcing reciprocity, leading to a form of asymmetric clustering. This approach not only enhances data utilization across the devices, but also maintains the integrity of high-quality data.
1. The paper presents a clustered federated learning method; however, it lacks an introduction to the implementation of federated learning (FL) and clustered federated learning (CFL). I recommend including details about the specific training processes and objectives of both FL and CFL in the BACKGROUND section to enhance clarity and context.

2. How can the proposed approach be effectively implemented in real-world scenarios? In practical federated learning settings, it seems challenging to determine whether cluster C2 supports cluster C1. Given that devices do not have visibility into each other's data during training, the primary method for assessing support appears to require multiple rounds of training and testing. This approach could lead to considerable resource inefficiencies.

3. The main operations of the method are presented in algorithmic form in the appendix (e.g., Algorithms 5 and 6). However, it may be more beneficial for readers to understand these concepts if they are described in text form within the main body of the paper.

4. In the experimental setup, there is no detailed information on how the data is divided or the rationale for this division. Different data partitioning methods can impact the final performance of federated learning.
Yes
No ethics review needed.
Please see the weakness.
The author provides a theoretical analysis of its proposed cluster strategy.
This paper presents a clustered federated learning method named FAACL, which aims to adaptively cluster devices and dispatch models for each device according to its cluster.
This paper seems to be an incremental work of CFL. The contribution is limited.

This paper is poorly written. E.g., 1) the author should add a figure of framework and workflow. 2) The expression of lines 186-196 seems to detail the contributions and steps of the proposed method, rather than the components. I advise the author to reorganize the expressions.

The author attempts to utilize CFL-based strategy to address the problem of non-IID data. However, the author lacks comparison the proposed method with SOTA tradition federated learning optimization methods [1-4].

The proposed method cannot adapt to the secure aggregation method. The author should analyze the privacy of the proposed method.

The experiments are mainly based on MNIST and its variants. I would like to know how the method performs on more complex datasets, such as ImageNet. Please note that the experimental results show that the proposed method is not effective enough on the cifar10 dataset and is even inferior to the baseline.
@inproceedings{
anonymous2024faacl,
title={{FAACL}: Federated Adaptive Asymmetric Clustered Learning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=gNrNlhhCN1},
note={under review}
}
Asymmetric clustering has remained an unexplored problem in Clustered Federated Learning (CFL), diverging from the traditional approach of forming independent, non-interacting clusters. Previous methodologies have been limited to either separating devices with different data quality into distinct clusters or merging all devices into a single cluster, both of which compromise either data utilization or model accuracy. We propose a new federated learning technique where some devices may contribute to the training of the models of other devices, but without enforcing reciprocity, leading to a form of asymmetric clustering.  This is beneficial in a variety of situations including scenarios where it is desirable for a device with high quality data to help train the model of a device with low quality data, but not vice-versa. This method not only enhances data utilization across the devices, but also maintains the integrity of high-quality data. Through a rigorous theoretical analysis and empirical evaluations, we demonstrate that our approach can efficiently find high quality (asymmetric) clusterings for numerous devices, achieving competitive performance metrics on existing CFL benchmarks.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Federated Learning Distributed Learning Clustering
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/54c6a0d4a10fd32f82816910841988e30ebe8f34.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
FAACL: Federated Adaptive Asymmetric Clustered Learning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission