MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning
Machine Unlearning;Benchmark;Multimodal Learning
This paper contains a GitHub link that reveals the identity of the authors. This broke double blind review.
Submission Desk Rejected by Program Chairs
We propose the first comprehensive multitask multimodal benchmark for machine unlearning (MU), including limitedly studied tasks and modalities, together with a series of base models to standardize MU evaluation.
@misc{
cheng2024mubench,
title={{MU}-Bench: A Multitask Multimodal Benchmark for Machine Unlearning},
author={Jiali Cheng and Hadi Amiri},
year={2024},
url={https://openreview.net/forum?id=O9W9DesXid}
}
Recent advancements in Machine Unlearning (MU) have introduced solutions to selectively remove certain training samples, such as those with outdated or sensitive information, from trained models. Despite these advancements, evaluation of MU methods have been inconsistent, employing different trained models and architectures, and sample removal strategies, which hampers accurate comparison. In addition, prior MU approaches have mainly focused on {\em singular} tasks or modalities, which is not comprehensive. To address these limitations, we develop \method, the first comprehensive benchmark for MU that 
\emph{(i) unifies the sets of deleted samples and trained models}, and
\emph{(ii) provides broad coverage of tasks and data modalities}, 
including previously unexplored domains such as speech and video classification. 
Our evaluation show that RandLabel and SalUn are the most effective general MU approaches on MU-Bench, and BadT and SCRUB are capable of achieving random performance on the deletion set. 
We analyze several under-investigated aspects of unlearning, including scalability, the impacts of parameter-efficient fine-tuning and curriculum learning, and susceptibility to dataset biases. 
MU-Bench provides an easy-to-use package that includes dataset splits, models, and implementations, together with a leader board to enable unified and scalable MU research\footnote{Code: \href{https://github.com/CLU-UML/MU-Bench}{https://github.com/CLU-UML/MU-Bench}}.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Jiali_Cheng1 ~Hadi_Amiri1
Jiali Cheng Hadi Amiri
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Machine Unlearning Benchmark Multimodal Learning
I certify that there is no acknowledgement section in this submission for double blind review.
cheng|mubench_a_multitask_multimodal_benchmark_for_machine_unlearning
/pdf/6face7344ec7e8461d4a44096acda72e3deca891.pdf
datasets and benchmarks
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning
ICLR 2025 Conference Desk Rejected Submission
ICLR.cc/2025/Conference/Desk_Rejected_Submission