Improved Localized Machine Unlearning Through the Lens of Memorization
Machine Unlearning;Memorization;Localized Unlearning
Yes
No ethics review needed.
- Please address the concerns in the weakness section.
- I liked the initial idea of investigating localized unlearning based on memorization. 
- The proposed method was partially successful on some forgetting benchmarks.
This work attempted to tackle the problem of localized unlearning by investigating it based on the memorization assumption and proposed DEL for some parameters with resetting and fine-tuning. The proposed method showed promising results on forgetting on a couple of benchmarks.
- The method is based on a lot of assumptions without much justification, but with intuition. Thus, it is very hard to see if the proposed method is indeed ok in terms of unlearning (while preserving the rest!). 
- It is very hard to see the core contribution clearly due to poor writing. It was very hard to read and follow.
- Experiments look quite limited in terms of benchmarks (datasets, compared methods). I am afraid that the localized unlearning approach may hurt the preservation of remaining parts, but it is unclear if it is true.
Yes
No ethics review needed.
Q1: From the Appendix A.4's algorithmn, the localization strategy is mainly from the magnitude of each weighted gradient for each mini-batch. Is the localization mask determined by each mini-batch? Is the localization mask fixed for different networks? If the mask is not accurate, does it affecting the accuracy? How sensitive is DEL to different choices of localization strategy.

Q2: Does the DEL method has any specific limitations when facing more complex or diverse data distributions?

Q3: Can DEL method adapted to other network architectures? What's the differences if it adapted to a customized network structure?

Q4: Does the performance different if using different hyper-parameters, such as learning rate, batch size, etc?

Q5:  In Table 7, the accuracy is getting better with higher percentage of parameters. Will the accuracy still getting better with 40%/50%?
This method achieves state-of-the-art unlearning performance while requiring a small modification on a subset of model parameters.

This method also minimized unnecessary parameter while preserving the model efficiency.
This paper introduced Deletion by Example Localization (DEL) method, which aimed at enhancing the machine unlearning by focusing on localized, a targeted data subset in neural networks. The traditional unlearning methods are removing the influence of certain data, making the model performance worse or requiring extensive retraining. However, DEL method used a selective approach by identifying a small subset of parameters that influenced by specific data points. This method can effectively remove the memory of specified data subset while persevering the model accuracy.
The weakness of this method is limited experiments on the public dataset, only applied on CIFAR-10 and SVHN datasets, as well as the limitation on larger models.
Yes
No ethics review needed.
1. I think the memorization property can vary from model scale. So, I am wondering if this memorization and proposed algorithm is available for most models since the evidence provided is empirical findings.
1. The localized unlearning is new and meaningful research area and the motivation to leverage the memorization is reasonable and insightful.

2. The experiments and findings are validated with various metrics and existing unlearning algorithms and show consistently good results.

3. The paper is well formatted and arranged so that easy to understand.
The paper proposes the local unlearning algorithm Deletion by Example Localization, leveraging the memorization issue. The proposed algorithm first resets the parameters that are most critical based on the localization strategy and then finetunes them. The algorithm can be paired with various existing unlearning algorithms. The author validates experiments on different datasets with different metrics to show that the performance achieves state-of-the-art.
1. There are several mathematical definitions such as the Unlearning and Label memorization. However, I did not find close connections or logical relations between them. If necessary, I expect the author to use these definitions to derive some theorems closely based on the proposed algorithm. For example, it is difficult to see theoretically or empirically if the proposed algorithm can make distribution the same as the model trained without that data.

2. Following the above, I understand in this area, most justifications are more empirical. So, I think it's better to use some metrics that can support the definition (I.e., the same distribution as a retrained model).
Yes
none
No ethics review needed.
In Section 5.1, your paper presents several hypotheses. Could you provide a more detailed explanation of how your results support these hypotheses?
1.	The paper is well-written, with a clear and concise logical flow. It begins by introducing localization as a preferred approach for model unlearning and then presents a cohesive and insightful perspective—that unlearning can be viewed as an extreme form of no memorization (lines 165-169)—which lends coherence and unity to their proposed method.
2.	The paper provides a comprehensive review of existing methods, thoroughly examining current approaches and establishing its own assumptions, such as the advantages of data-dependent over data-agnostic methods and the reasoning for utilizing gradient information. These insights serve as the foundation for their proposed method, DEL.
3.	The proposed method is both simple and effective, achieving state-of-the-art performance.
This paper addresses the challenge of machine unlearning in a localized context by introducing a novel approach based on the concept of memorization. Following a comparison of existing methods, the authors identify data-dependent and gradient-dependent techniques as particularly effective. They refine the current criticality-based localization strategy, resulting in a new unlearning algorithm, “Deletion by Example Localization” (DEL). DEL enables localized unlearning by resetting and fine-tuning parameters identified as essential based on the calculated criticality of the parameters.
1. This paper extensively discusses related work and motivations, primarily focusing on comparisons between existing methods. The proposed approach appears to be a straightforward combination of existing techniques, which may limit its novelty.
2. The results in Section 3 do not necessarily support the hypotheses in Section 5.1, as the observed improvements could be attributed to other factors. Thus, a more thorough theoretical explanation of the proposed method is needed.
3. This paper focuses exclusively on classification models, but I believe that “unlearning” in LLMs (i.e., model or knowledge editing) is a more pressing concern. It remains uncertain whether the conclusions drawn from vision classifiers in this paper can be directly applied to LLMs.
4. There are a few typos, although they don’t impact comprehension. For instance, in line 159, “$f(; \theta)$” might be intended as “$f(x; \theta)$.”
@inproceedings{
anonymous2024improved,
title={Improved Localized Machine Unlearning Through the Lens of Memorization},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=2m5XI3nM46},
note={under review}
}
Machine unlearning refers to removing the influence of a specified subset of training data from a machine learning model, efficiently, after it has already been trained. This is important for key applications, including making the model more accurate by removing outdated, mislabeled, or poisoned data. In this work, we study localized unlearning, where the unlearning algorithm operates on a (small) identified subset of parameters. Drawing inspiration from the memorization literature, we propose an improved localization strategy that yields strong results when paired with existing unlearning algorithms. We also propose a new unlearning algorithm, Deletion by Example Localization (DEL), that resets the parameters deemed-to-be most critical according to our localization strategy, and then finetunes them. Our extensive experiments on different datasets, forget sets and metrics reveal that DEL sets a new state-of-the-art for unlearning metrics, against both localized and full-parameter methods, while modifying a small subset of parameters, and outperforms the state-of-the-art localized unlearning in terms of test accuracy too.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Machine Unlearning Memorization Localized Unlearning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/150921e5d33db3b86c10e698cedf4711f038cb96.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Improved Localized Machine Unlearning Through the Lens of Memorization
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission