$\lambda$-SecAgg: Partial Vector Freezing for Lightweight Secure Aggregation in Federated Learning
Secure aggregation;Federated learning
Yes
No ethics review needed.
Please see the comment in the weaknesses.
1. This paper considers a timely and important problem in secure aggregation protocol to reduce computational and communication overhead.
This paper introduces $\lambda$-SecAgg, a secure aggregation protocol for federated learning (FL) designed to reduce computational and communication overhead through Partial Vector Freezing (PVF). This paper claims that by freezing and processing only a fraction of the private vector entries, the method significantly reduces the burden on the server and participating devices while ensuring all vector entries are eventually aggregated. To further enhance privacy, the paper incorporates Disrupting Variables Extension (DVE). The authors empirically demonstrate substantial performance gains in terms of speedup and communication reduction across various secure aggregation protocols.

While the paper presents an interesting method to reduce the overhead in secure aggregation, the privacy analysis in Section 4.1 is fundamentally flawed. The authors underestimate the information leakage from $y^{i}$, which compromises the claimed privacy guarantees.
1. Most importantly, the privacy analysis in Section 4.1, which claims no privacy leakage from $y^{i}$, is flawed. Although the paper asserts that no specific element of the original vector $x$ can be deduced directly from $y^{i}$, this does not mean there is no privacy leakage. In fact, $y^{i}$ reveals significant information about $x$. For example, in the case where $\lambda = 2$ and $x$ has two elements, the server can infer $x_1$ in terms of $x_2$ from $y^{i} = a_{11}x_1 + a_{12}x_2$. While $x_1$ cannot be fully determined without $x_2$, the conditional probability of guessing $x_1$ correctly is now $1/p$ instead of $1/{p^2}$. This reduction in entropy, $H(x)$, shows that $y^{i}$ contains valuable information, thus reducing privacy. The authors should revise the privacy analysis and clarify the impact of knowing $y^{i}$ on the security of the original vectors.
Yes
No ethics review needed.
1. The baseline runtime figures for the secure aggregation protocols presented in Figure 1 and Table 1 appear notably higher than those reported in related literature. For instance, in the case of PracAgg with a vector length of 100k elements, Figure 1 shows a client runtime of 14 seconds and a server runtime of 140 seconds. In contrast, the original paper by Bonawitz et al. (2017) reports significantly lower runtimes for similar conditions, with client runtimes around 300 milliseconds (Figure 6a) and server runtimes at most 5 seconds (Figure 7a). Could you clarify the reasons for this discrepancy in runtime comparisons?
2. Could you provide a more detailed analysis of the privacy impact of your scheme, particularly focusing on the amount of differentially private noise that would be sufficient to mitigate privacy risks effectively? A clearer discussion on how the noise level was determined and its implications on both privacy and utility would be valuable.
- The focus of privacy-preserving federated learning is a crucial topic
- Extensive evaluation that covers a wide range of existing secure aggregation protocols
The authors present a new system to improve the computational overhead of secure aggregation using a new approach called Partial Vector Freezing. This approach reduces the number of entries processed in the secure aggregation protocol, by projecting chunks of the client input vector onto a different space, and only aggregating $1/\lambda$ of the entries of each chunk securely and sending the rest of the entries in the clear. The server aggregates the entries from all clients and recovers the original input vectors by projecting the inputs back to the original space. The paper further bolsters privacy through the Disrupting Variables Extension, which applies noise calibrated for Local Differential Privacy to frozen vectors. Experimental results demonstrate substantial computation improvements compared to state-of-the-art secure aggregation protocols.
- The approach impacts the robust privacy guarantees traditionally upheld by state-of-the-art secure aggregation protocols. These protocols typically ensure that an adversary gains no additional information about the inputs of honest clients beyond what is inferred from the aggregated output. However, Partial Vector Freezing (PVF) significantly reduces this privacy. As pointed out by the authors, it is possible for the server to learn whether two clients have similar vector chunks. Although the authors propose a mitigation strategy through Local Differential Privacy to reduce the detection of exact matches, this measure does not fully mitigate the issue of input privacy. The noised client inputs may still leak partial information that allows the server to deduce similarities between inputs. Given this trade-off, the computational gains provided by PVF do not justify the notable privacy impact. For instance, in the context of PracAgg, the masking computation is relatively lightweight. It involves field operations and pseudorandom generator evaluations, typically implemented with efficient cryptographic functions like AES. Additionally, the more computationally intensive pairwise key agreements are independent of the vector size and remain necessary regardless of the implementation of PVF.
- Another concern is the soundness of the security proof presented in Theorem 1. Specifically, the claim that the protocol execution is indistinguishable from random simulation seems to be inaccurate. The distribution of Hybrid 1 is not indistinguishable from that of Hybrid 0, as the distribution of frozen vectors $y_i$ does not exhibit properties of uniformly sampled vectors. While the random vectors are sampled uniformly from $\mathbb{Z}_p$, the frozen vectors in the protocol are the actual inputs masked with centered Gaussian noise of bounded variance. This results in a non-uniform distribution over $\mathbb{Z}_p$ undermining the indistinguishability between the two hybrids. Furthermore, other parts of the security proof are incomplete.  For instance, in Hybrid 3, it is stated that the adversary-controlled clients $\mathcal{C}$ call the ideal functionality. However, in simulation-based proofs, it is typically the simulator, not the adversary, that has direct access to the ideal functionality. Clarifying this aspect would strengthen the proof’s rigor and ensure alignment with standard cryptographic practices.
Yes
No ethics review needed.
This paper presents a novel and practical approach to reduce the computational overhead of secure aggregation in federated learning by proposing λ-SecAgg with partial vector freezing (PVF). The strengths of the method lie in its innovative design, theoretical rigor and comprehensive evaluation, showing significant performance improvements. However, there are areas that could benefit from further clarification, particularly in terms of readability, real-world evaluation, and the impact of noise on accuracy. It is recommended that the authors consider the above comments to further refine and optimize the paper.
Innovation: The concept of freezing and unfreezing vector entries in the context of secure aggregation is very novel. This approach effectively reduces the computational burden on SAP, which has been a significant bottleneck in real-world federated learning applications, especially for large-scale models such as Large Language Models (LLMs).

Comprehensive Evaluation: The authors evaluate their approach on seven different baselines covering various secure aggregation protocols (e.g., homomorphic encryption-based, SMPC-based, mask-based). The experimental results show substantial improvements in computation time and communication cost.

Privacy and Security: The paper proves the privacy guarantees of λ-SecAgg under semi-honest and active adversary models through security analyses. In addition, the authors introduce extensions such as DVE, which further strengthens the privacy guarantees.
This paper introduces a novel method called λ-SecAgg, which integrates a module named Partial Vector Freezing (PVF) into Secure Aggregation Protocols (SAPs) for federated learning. The main goal of this method is to reduce the computational overhead by “freezing” most of the entries in user update vectors, allowing only a fraction (1/λ) of the original vector to be processed through secure aggregation. The frozen entries can later be “thawed” to recover the full aggregated vector, ensuring that no information is lost in the final aggregation. Additionally, the paper proposes a Disrupting Variables Extension (DVE) that enhances privacy by adding noise to the frozen entries using Differential Privacy (DP). The authors perform extensive empirical evaluations across seven baselines, demonstrating that PVF can achieve up to 99.5× speedup and 32.3× communication reduction without compromising user privacy or security.
Clarity and readability: Although this paper presents a novel approach, some sections are dense and difficult to understand, especially the mathematical derivations and safety analyses. It is suggested that the authors could improve the readability of these sections by providing more intuitive explanations and breaking down the steps as much as possible. In addition, the readability of some diagrams and formulas (e.g., those in Sections 3 and 4) is too low, and it is suggested that the reader could improve them by simplifying them or providing more detailed explanations.

Impact of noise on accuracy: Although the paper claims that the impact of DVE (adding noise to DP) on accuracy is negligible, the experimental results on the loss of accuracy due to DP noise are not detailed enough. It is suggested that the authors can add relevant experiments for this part.

Scalability to Multiple Users: This paper focuses on performance improvements for single users and servers, but does not discuss scalability to multiple users. It is suggested that the authors validate the approach of this paper in the context of multiple simultaneous users, especially with respect to communication overheads and system latency.
Yes
No ethics review needed.
1. In practical applications, how should this value \lambda be determined?

2. Are there any fundamental differences between the aggregation method of k^i and that of y^i?
1. The PVF significantly compresses the length of the vector involved in SAP.
2. The disrupting variables extension method improves privacy, without the computational overhead.
3. The authors conduct extensive experiments
The paper addresses the challenges of secure aggregation in federated learning, particularly the high computation costs associated with Secure Aggregation Protocols (SAPs). The paper introduces a novel approach called Partial Vector Freezing (PVF), designed to reduce computation without increasing communication overhead. In addition, the paper proposes the disrupting variable extension to PVF to support enhanced privacy. The extensive experiments show the effectiveness of the proposed proposal.
1. Lack of Novelty in the Proposed Solution:
While I appreciate the clarity and straightforwardness presented in your methodology, I am concerned about the apparent simplicity of the proposed solution. The approach, as described, seems to lack the level of innovation. Consider expanding on the theoretical background, comparing your method with others in detail, and emphasizing any novel insights or improvements that your solution offers.
2. Informality in Security Analysis:
The security analysis section of your paper appears to be somewhat informal and lacks the rigor typically required for a comprehensive evaluation of a proposed system or method. Security is a critical aspect in many research domains, and a thorough, formal analysis is essential to establish trustworthiness and robustness. I recommend conducting a more structured and detailed security analysis, possibly incorporating formal security proofs, case studies, or simulations to demonstrate the effectiveness of your security measures.
Yes
No ethics review needed.
Please refer to the weakness above.
1.Theoretical proofs.
2.The experimental results demonstrate that PVF achieved 99.5 \times speedup and up to 32.3 \times communication reduction.
This paper devises a portable module named \lamba-SecAgg for secure aggregation in federated learning. The authors also propose an extension involving disrupting variables to enhance privacy. Through extensive experiments, they demonstrate the efficiency of the proposed method, achieving up to 99.5× speedup and up to 32.3× communication reduction.
1. Writing/technical issues:
(1) In the Introduction section, the author methioed that "the minimal noise added by DP is insufficient to thwart attacks", yet they also suggest considering "DP in the extension for enhanced privacy." Is there a deeper reason or gap that I might have overlooked?
(2) The introduction of "compression-based techniques" in Figure 3 and Section 2 feels somewhat abrupt, primarily due to the lack of clarity in the classification of existing solutions outlined in the Introduction section. I suggest providing a clearer explanation of the criteria used to categorize the methods into secure aggregation techniques and compression-based techniques. Additionally, providing an analysis of the limitations of existing methods would help readers better understand the motivation behind the development of the PVF.
(3) The definition of adversary in threat model is not very clear, particularly regarding key aspects such as adversary knowledge and adversary capabilities, which have not been adequately explained or defined.
(4) Figure 4 is too abstract to understand.
(5) In Section 3.3, while discussing secure aggregation, it is noted that the requirements for data accuracy are relatively high. However, the introduction of DP typically involves adding noise to the data. It would be beneficial to clarify how the accuracy of the data can be maintained after noise has been added, particularly in the context of the freezing and melting processes.

2. Experimental issues:
(1)The neural network architectures and datasets are not intruduced in the ‘Experimental settings’.
(2)The setting of (\lambda = 100) in some experiments requires further explanation.
(3)The experimental validation, although comprehensive, is limited to specific neural network architectures and datasets. Its generalisability to other models and types of data may require further examination.
Yes
No ethics review needed.
Could please you address the points raised in "Main Weaknesses" above? 

In addition to these questions: 
- Could you illustrate in more detail what which masking operations of the compared SecAgg protocols do you avoid by the use of your proposal? It seems that either performing a matrix-vector multiplication or masking operations does not eliminate the dependency of the dimension of a vector $x$ in the computation. 

- If we compare the computational cost of the SecAgg protocol and the computational cost of a local training step for a client, what proportion of the computation the SecAgg overhead represents? How does this change for different models?
- Reducing the communication cost in privacy preserving ML an interesting topic.

- The presentation of the protocol is fairly clear.
The paper studies how to reduce the computational cost in privacy-preserving federated learning (FL) with secure aggregation (SecAgg). SecAgg is a primitive that improves the privacy-utility trade-off in FL as it hides individual model updates sent to the server. However, most efficient applications require to mask each parameter of the model with random noise, incurring into a large computational cost if models are big. 

The current work proposes a technique that reduces the computational cost by only performing SecAgg to a subset of model parameters, while still recovering the (claimed to be) private aggregation of the entire model.
# Main Weaknesses 

The major weakness of the protocol is the **lack of any standard notion of security**. The protocol is based on the fact that revealing the undetermined system of linear equations $\breve{A}x = y$ where $\breve{A}$ and $y$ are public does not compromise the privacy of $x$. From a security point of view, letting the adversary gain the knowledge of $\breve{A}x$ is **completely unsafe**. A clear example is already given the detailed comments subsection below for certain choices of $\breve{A}$. 

Even if the paper proposes some defenses to avoid the most obvious threats in the choice of $\breve{A}$ (i.e. if the system of equations already directly exposes some coordinates of $x$), these defenses are only a minor improvement in the overall security. The protocol is in fact insecure for any $\breve{A}$. For example, consider that a party joins the aggregation protocol with a vector $x = (x_1, 0, \dots, 0)$ (i.e., a vector where $x_1$ is the only non-zero value). In this case, values of $\breve{A}x$ will always be multiples of $x_1$. Therefore, the claim of Theorem 1 does not hold: multiples of $x_1$ are *distinguishable* from uniformly random numbers, contrary to what is claimed in hybrid 1 ($H_1$) in the proof of Theorem 1 (Appendix D.1).  This renders the proof of Theorem 1 incorrect. 

The computational improvements of this protocol come from the insecure modification described above. This makes the protocol inapplicable. Moreover, the attempts to further "complicate" the linear equations by the presented enhancements also follow an unsafe methodology lacking proper proofs (see my detailed comments below). 

In addition to the above, the work ignores important lines of work in compression under privacy constraints (e.g. see [R1-R5] below) and differentially private-based aggregation (e.g., by the use of correlated noise [R6-R8]), directly related to the current contribution. 


# Detailed Comments 

- Page 2, Section 2: 
    - "Mask-based" approaches are an instantiation of "SMPC-based" approaches. 
    - "(i) improving the masking mechanism":  it is not clear what this means 
    - "Note that the security of FL remains an open issue": this is too broad and it is not clear what "security" means in this context
- Page 3: 
    -  Section 2: "However, their ability to prevent poisoning attacks is limited (Ma et al., 2023)": not sure how the reference is relevant here. Does  (Ma et al., 2023) provides evidence about this statement? 
    - Section 3,  "ultimately imposing significant computational burdens": "burdens" $\rightarrow$ "burden"; is this computational burden significant with respect to the computational cost of local training steps required by ML? 
- Page 4: 
    - Def 1: "... where $AK$ denotes the additional knowledge ..": so far no mention of "additional knowledge has been made", so it is not clear to what this refers. Also, it should be explicitly clarified that $rank(A, Ax)$ means the rank of the horizontal concatenation of $A$ and $Ax$. 
   - "... rendering it impossible to determine that specific confidential vector." this is an overly strong statement (at least if no additional context is given). Consider for example that $A$ equals the identity matrix. Indeed $\breve{A}x$ has infinite solutions (all possible values of the removed coordinate of $x$). However, almost all coordinates of $x$ will be revealed if $\breve{A}x$ is revealed. 
- Page 5, Sec 3.3: 
    -  "... even if some secagg entries are compromised": There is no motivation of the extra defense, explaining how these entries would be compromised. 
    - "complicating the relationships among entries and further enhancing privacy": This lacks a proof. Providing privacy by obscurity (i.e, providing an obfuscation technique without proving that indeed it reduces to hard problem for the adversary) is a bad practice in the field of security.


# References 

[R1] Bassily, Raef, and Adam Smith. “Local, Private, Efficient Protocols for Succinct Histograms.” In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing, 127–35. STOC ’15. New York, NY, USA: Association for Computing Machinery, 2015. https://doi.org/10.1145/2746539.2746632.

[R2] Feldman, Vitaly, and Kunal Talwar. “Lossless Compression of Efficient Private Local Randomizers.” In Proceedings of the 38th International Conference on Machine Learning, 3208–19. PMLR, 2021. https://proceedings.mlr.press/v139/feldman21a.html.

[R3] Liu, Yanxiao, Wei-Ning Chen, Ayfer Özgür, and Cheuk Ting Li. “Universal Exact Compression of Differentially Private Mechanisms.” arXiv, May 28, 2024. https://doi.org/10.48550/arXiv.2405.20782.

[R4] Shah, Abhin, Wei-Ning Chen, Johannes Ballé, Peter Kairouz, and Lucas Theis. “Optimal Compression of Locally Differentially Private Mechanisms.” In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, 7680–7723. PMLR, 2022. https://proceedings.mlr.press/v151/shah22b.html.

[R5] Triastcyn, Aleksei, Matthias Reisser, and Christos Louizos. “DP-REC: Private & Communication-Efficient Federated Learning.” arXiv, December 7, 2021. https://doi.org/10.48550/arXiv.2111.05454.

[R6] Imtiaz, Hafiz, Jafar Mohammadi, and Anand D. Sarwate. “Distributed Differentially Private Computation of Functions with Correlated Noise.” arXiv, February 22, 2021. https://doi.org/10.48550/arXiv.1904.10059.

[R7] Kairouz, Peter, Brendan Mcmahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, and Zheng Xu. “Practical and Private (Deep) Learning Without Sampling or Shuffling.” In Proceedings of the 38th International Conference on Machine Learning, 5213–25. PMLR, 2021. https://proceedings.mlr.press/v139/kairouz21b.html.

[R8] Sabater, César, Aurélien Bellet, and Jan Ramon. “An Accurate, Scalable and Verifiable Protocol for Federated Differentially Private Averaging.” Machine Learning 111, no. 11 (November 1, 2022): 4249–93. https://doi.org/10.1007/s10994-022-06267-9.
A plug-and-play module to compress computation overhead in secure aggregation protocols for federated learning, requiring only $\frac{1}{\lambda}$ of the original vector to participate in the protocol.
@inproceedings{
anonymous2024lambdasecagg,
title={\${\textbackslash}lambda\$-SecAgg: Partial Vector Freezing for Lightweight Secure Aggregation in Federated Learning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=E1Tr7wTlIt},
note={under review}
}
Secure aggregation of user update vectors (e.g. gradients) has become a critical issue in the field of federated learning. Many Secure Aggregation Protocols (SAPs) face exorbitant computation costs, severely constraining their applicability. Given the observation that a considerable portion of SAP's computation burden stems from processing each entry in the private vectors, we propose Partial Vector Freezing (PVF), a portable module for compressing computation costs without introducing additional communication overhead. $\lambda$-SecAgg, which integrates SAP with PVF, "freezes" a substantial portion of the private vector through specific transformations, requiring only $\frac{1}{\lambda}$ of the original vector to participate in SAP. Eventually, users can "thaw" the public sum of the "frozen entries" by the result of SAP. To enhance privacy, we devise Disrupting Variables Extension for PVF. We demonstrate that PVF can seamlessly integrate with various SAPs and it poses no threat to user privacy in the semi-honest and active adversary settings. We include $7$ baselines, encompassing $5$ distinct types of masking schemes, and explore the acceleration effects of PVF on these SAPs. Empirical investigations indicate that when $\lambda=100$, PVF yields up to $99.5\times$ speedup and up to $32.3\times$ communication reduction.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Secure aggregation Federated learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/6aa219dd5e037007d2d29b20c46d1cf7a874256f.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
$\lambda$-SecAgg: Partial Vector Freezing for Lightweight Secure Aggregation in Federated Learning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission