Interaction Based Gaussian Weighting Clustering for Federated Learning
Federated Learning;Clustered Federated Learning;Personalized Federated Learning
Yes
No ethics review needed.
See weakness.
- Comprehensive experiments
- mathematical analysis.
To address the heterogeneity and class imbalance issues in FL, this paper proposes FedGWC, a novel PFL method. Further experiments show that FedGWC outperforms existing FL methods. The clients are grouped into clusters with similar data distribution based on an interaction matrix.
- This paper is hard to follow, especially in the methodology part. e.g, the description of Eq(1). It is also unclear how we get the clusters based on Algorithm 1.
- What’s the meaning of the dash line in Figure1 and how we get the average loss process.
- The relation between $L_k^{t,s}$ and $l_k^{t,s}$.
- What if some clients are hardly sampled, does that affect his reward estimate?
- In the line 259, “relying only on the values of the Gaussian weights is insufficient to identify clusters of clients with similar data distributions, as they do not capture the interactions among pairs of clients.” However, it is not clear why we need the interactions among pairs of clients. I can not catch the motivation.
- What is the clients’ sampling rate at each round?
- It is not applicable that this method needs 20000 communication rounds to be converged.
- Extra communication and computation overhead analysis is needed to compare with baselines.
Yes
No ethics review needed.
I would be appreciated if the authors can address my concerns listed in the weaknesses, including the communication efficiency, privacy, scalability, and more comprehensive experiments.
1. FedGWC introduces a Gaussian weighting method that offers a fresh approach to clustering by leveraging interaction-based weights instead of relying on traditional model updates. 
2. The proposed method addresses significant FL challenges like non-IID data and class imbalance, showing strong adaptability in scenarios with high heterogeneity.
3. The FedGWC algorithm can be applied with various FL aggregation algorithms (e.g., FedAvg, FedProx) and performs well even with personalized FL techniques like pFedMe and Per-FedAvg.
4. The paper provides a mathematical foundation for the convergence and consistency of the Gaussian weights, adding robustness to the proposed method.
In this paper, the authors propose a novel PFL method named FedGWC (Federated Gaussian Weighting). FedGWC groups clients based on their data distribution, allowing training of a more robust and personalized model on the identified clusters. The experimental results have shown the effectiveness of the proposed method and its versatile integration to various FL aggregation algorithms.
1. Although the algorithm mitigates communication overhead, the interaction matrix and clustering computations may still introduce complexity, especially in large-scale FL deployments with a high number of clients.
2. The paper doesn’t extensively discuss the implications of FedGWC’s clustering on privacy, especially since clustering-based methods might reveal distribution characteristics indirectly.
3. The clustering results are influenced by parameters such as the RBF kernel spread, which could require extensive tuning. This dependency may limit FedGWC’s practicality across different datasets without manual optimization.
4. The empirical validation is limited to specific datasets (CIFAR10, CIFAR100, and FEMNIST). Broader evaluations on diverse datasets, including real-world applications, could strengthen the practical applicability of the findings.
Yes
No ethics review needed.
See the weaknesses.
1. The FedGWC method effectively addresses the challenge of data heterogeneity and class imbalance by leveraging a novel clustering-based approach.

2. The introduction of a new clustering metric specific to federated learning provides a valuable tool for assessing clustering quality in class-imbalanced environments.
This paper proposes a personalized federated learning (PFL) framework called FedGWC (Federated Gaussian Weighting Clustering), which groups clients based on data similarity using Gaussian weighting on empirical losses. This clustering technique is designed to address data heterogeneity and class imbalance by forming homogeneous clusters, where each cluster benefits from personalized federated models trained on data with similar distributions. The authors introduce a clustering metric for evaluating cohesion among clients and conduct experiments that demonstrate FedGWC's performance gains over baseline methods in heterogeneous data scenarios.
1. The clustering methodology relies on empirical losses, which may not fully capture data distribution nuances across diverse client datasets, potentially leading to suboptimal clustering in more complex scenarios.

2. The computational cost of clustering based on interaction matrices and Gaussian weighting could limit scalability, especially in large federations with numerous clients and communication rounds.

3. The method's convergence properties, though theoretically addressed, lack extensive empirical validation in highly non-IID settings, making its performance in real-world FL scenarios uncertain.
Yes
No ethics review needed.
1. This work is inspired by (Cho et al., 2022) to capture the similarity in client data distributions through a transformation of the loss function. Please further explain why measuring loss function distance can “effectively identify clusters of clients with similar levels of heterogeneity and class distribution.”

2. What is the significance of the evaluation metric introduced based on the Wasserstein distance, and is there any further improvement to FedGWC based on this metric?
1. This work uses the similarity of loss functions as the basis for clustering and provides a theoretical proof of the convergence of the optimization algorithm using Gaussian weights.

2. The proposed method, FedGWC, is orthogonal to other aggregation methods.

3. The algorithm’s performance is examined under imbalanced scenarios.
This work proposes a federated learning clustering strategy based on loss function similarity and introduces a clustering metric based on Wasserstein distance. The paper provides a theoretical proof of the algorithm’s convergence and demonstrates its performance in various imbalanced scenarios.
1. The study lacks recent federated learning clustering baselines. The three comparison methods are [CFL, 2020], [FeSEM, 2023], and [IFCA, 2020]. Moreover, the proposed FedGWC performs weaker than other methods on CIFAR-10 and FEMNIST datasets and is similar to FeSEM on CIFAR-100. Overall, it does not achieve superior performance.

2. The comparative experiments in Table 3 are unfair, as FeSEM is selected across all three datasets. Instead, the best baseline should be selected for each dataset: IFCA for CIFAR-10, FeSEM for CIFAR-100, and IFCA for FEMNIST.

3. The experiments in Table 5 are not convincing. Other baseline methods’ Rand Index scores should be added.

4. The design of Tables 2 and 3 could be optimized, as they are somewhat confusing.
We present a novel approach for clustering clients in FL to mitigate the effects of heterogeneity and class-imbalance within clients' data distribution. We motivate our work with a comprehensive mathematical framework
@inproceedings{
anonymous2024interaction,
title={Interaction Based Gaussian Weighting Clustering for Federated Learning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=JF8ovQyueq},
note={under review}
}
Federated learning emerged as a decentralized paradigm to train models while securing privacy. However, conventional FL faces data heterogeneity and class imbalance challenges, affecting model performance. In response to these issues, Personalized FL has been developed as an innovative methodology that relies on fine-tuning the distinct local models based on individual training datasets. In this work, we propose a novel PFL method, FedGW (Federated Gaussian Weighting), which groups clients based on their data distribution, allowing training of a more robust and personalized model on the identified clusters. FedGW identifies homogeneous clusters by transforming individual empirical losses to model client interactions with a Gaussian reward mechanism. Additionally, we introduce a new clustering metric for FL to evaluate cluster cohesion with respect to the individual class distribution. Our experiments on benchmark datasets show that FedGW outperforms existing FL algorithms in cluster quality and classification accuracy, validating the efficacy of our approach.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Federated Learning Clustered Federated Learning Personalized Federated Learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/93fe396c969f6a0f9c910d98bd1b9ac4c9ae7ed8.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/d559b0906fa5f8942a46fe76d61d958c7e7c9bdd.zip
Interaction Based Gaussian Weighting Clustering for Federated Learning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission