Auditing $f$-Differential Privacy in One Run
Differential privacy;Auditing privacy
Yes
No ethics review needed.
Questions:
* You claim that your approach achieves tighter results as the number of canaries increases, outperforming the empirical privacy results from Steinke et al. (2023), suggesting that the results can be tight as we increase the number of canaries. Could you elaborate on why your bounds continue to improve with more canaries while the bounds in previous work degrade? What underlying mechanisms in your algorithm contribute to this improvement? Citing the authors: ” Figure 1 demonstrates that our approach outperforms the empirical privacy results from Steinke et al. Interestingly, while the bound in Steinke et al. (2023) degrades as the number of canaries increases, our bounds continue to improve.”
* What potential sources contribute to any lack of tightness in your lower bounds? Are there specific aspects of the f-DP framework or your implementation that introduce looseness? How might these be addressed in future work to enhance the tightness of the bounds?
* How does your algorithm perform in the black-box setting compared to the white-box setting? Can you provide detailed experimental results illustrating this performance?
* **Advancement of f-DP Tools**: The paper contributes to the understanding and practical application of f-DP, which could be of independent interest.
* **Interesting Problem**: Auditing DP mechanisms in a one-run scenario is interesting for practical implementations (particularly in the black-box scenario, see the weakness section), and the paper makes significant progress in this area.
* **Experimental Validation**: The experimental results are compelling and demonstrate the effectiveness of the proposed approach.
* **Versatility in Adversarial Models**: Extending the auditing algorithm to handle different adversaries, such as reconstruction attacks.
The paper presents a novel algorithm for auditing differential privacy (DP) mechanisms in a single run, building upon and extending the work of Steinke et al. (2023). By leveraging the theoretical framework of f-Differential Privacy (f-DP), the authors provide tighter lower bounds on privacy leakage, thereby enhancing the existing toolbox for f-DP analysis. Notably, their auditing algorithm can be applied to various adversaries beyond the standard membership inference, such as reconstruction attacks.
The authors investigate exciting problems and provide interesting results. I encourage the authors to continue working on these results, as they are sound and exciting to the DP community. However, I don’t think the work is ready to be published in its current form, as it is somewhat rushed. I sketch my main concerns below.
* **Writing and Presentation Quality**: The manuscript contains several errors and unclear explanations. The authors should revise it before publication, as there are plenty of writing errors and bad citing style.
* **Unreferenced Figures and Results**: Some results, particularly those in Figure 7, need to be adequately referenced or explained within the text, leading to confusion about their significance.
* **Incomplete Explanation of Gaps**: The paper needs to explain the gaps between theoretical and lower bounds. Possible reasons for these gaps should be analysed, such as limitations of the f-DP framework, assumptions made in the analysis, or practical considerations in implementation.
* **Insufficient Experimental Details**: There are no experiments in the black-box setting for which we are compelled to use one-shot auditing. The white-box setting enjoys a tight and efficient auditing algorithm (Nasr et al., 2023), while the black-box algorithms are rather expensive.
Yes
No ethics review needed.
- On the section “Empirical Privacy” line no 307, why do the trade off curves need to be ordered? If you have a set of trade off curves $f_i$ that pass couldn’t you build a new trade off curve $f(x) = \min_i f_i(x)$ 
- In what sense are the empirical results tight in Fig 7 and why is that not also evident in Fig 1?
- Can you explain why abstentions are important in this algorithm?
- Valuable Contribution to Existing Research: There has been extensive work on auditing differential privacy guarantees. This paper distinguishes itself by offering a solution that enhances both computational efficiency and the precision of empirical privacy guarantees. The reliance on multiple runs of the mechanism has been a major obstacle to the widespread application of auditing methods. Their approach, requiring only a single run, makes auditing significantly more practical, especially for complex machine-learning algorithms involving extensive model training.
- Using the $f$-DP framework is a particularly strong aspect of this work. $f$-DP offers a more general and accurate representation of a mechanism's privacy compared to traditional approximate differential privacy. This choice allows for a more fine-grained and robust analysis of privacy. The authors convincingly demonstrate that auditing $f$-DP leads to tighter empirical privacy assessments. By performing the analysis in a single training run, the paper achieves a more comprehensive understanding of the privacy implications within a practical computational framework.
The paper presents a novel algorithm designed to audit $f$-DP guarantees within a single execution of a mechanism.
This area of research has become increasingly significant within the privacy community, particularly due to the limitations of existing auditing mechanisms.
Existing empirical auditing methods are either computationally expensive (requiring multiple runs of the machine learning algorithm) or fall short in providing a tight empirical privacy guarantee.
The need to run the mechanism multiple times has hindered practical applications.
Steinke et al. (2023) introduced a pioneering approach that balances the number of runs with the tightness of the audit.
This present work enhances this trade-off further by auditing $f$-DP guarantees, which provide a more precise representation of a mechanism's privacy compared to traditional approximate DP parameters.
- The main weakness of this paper is its presentation. The write-up seems very rushed which at times hinders the flow of the reader. Many references are broken e.g. reference to Algorithm B. Lines 300-312 contain many typos and incomplete sentences. These are issues that can be addressed quickly but in the current state I would argue that the presentations limits the value of this work to the community.
- The authors have not provided a code artifact. While the contributions of this work are mostly theoretical, the implementation of the algorithm requires care and it would help reproducibility if a code artifact were supplied.
Yes
No ethics review needed.
1) What do you mean by "gubernatorial analysis"? (Line 95)
2) Do you have an intuition why the bound in Steinke et al. (2023) degrades with higher numbers of canaries while your bounds continue to improve?
The paper is well-motivated and, for the most part, clearly written. It provides a notable improvement over prior privacy auditing techniques.
This paper proposes a computationally efficient privacy auditing procedure by leveraging the f-DP curve, and shows that the resulting lower bounds are tighter than those of previous work.
The paper contains some ambiguities and cosmetic errors that should be addressed to improve clarity and overall presentation.
1) clarify that by "one run" you mean a training run (rather than an inference run)
2) explicitly state the limitation of Steinke et al. (2023) that you are addressing (in Line 80-82)
3) change the references to algorithm 3.1 to algorithm 3 (given that that is what the algorithm is called)
4) remove double mentions of Steinke et al. by just using the reference instead (e.g., in Line 420)
5) fix the reading flow in Definition 6 (second bullet point is not a full sentence)
6) correct typos (e.g., Line 194/195, 307, 466, 505/506) and wrong capitalizations in the middle of sentences (e.g., Line 100)
We use trade-off functions to perform tighter auditing of algorithms designed to satisfy differential privacy in a single run.
@inproceedings{
anonymous2024auditing,
title={Auditing \$f\$-Differential Privacy in One Run},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=0QZcoGdmtJ},
note={under review}
}
Privacy-preserving machine learning requires carefully designed and rigorously analyzed algorithms. However, such designs and analyses are often susceptible to errors or imperfections, leading to mechanisms that may not offer the expected level of privacy due to mathematical inaccuracies or implementation flaws. Conversely, some mechanisms might provide stronger privacy guarantees than can be proven through a loose analysis. Empirical privacy auditing has emerged as a means to address this gap. Existing auditing mechanisms, however, are either inefficient—requiring multiple runs of machine learning algorithms—or suboptimal in calculating the empirical privacy of these algorithms. In this work, we present a tight and efficient auditing procedure and analysis that can effectively assess the privacy of mechanisms. Our approach requires only a single run of the mechanism and achieves tight empirical privacy by leveraging the $f$-DP curve, which provides a more accurate measure of privacy than the traditional $\epsilon,\delta$ parameters. Experiments demonstrate that our auditing algorithm delivers tighter empirical privacy guarantees.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Differential privacy Auditing privacy
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/803c0af3ab3dbae41b9b19d6db963d36fe59243e.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Auditing $f$-Differential Privacy in One Run
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission