Data Taggants: Dataset Ownership Verification Via Harmless Targeted Data Poisoning
dataset watermarking;dataset ownership verification;data poisoning;backdoor attack
Yes
No ethics review needed.
1. How does the method defend against an adversary who knows the exact verification technique?
2. Why was 0.1% chosen as the modification budget, and how sensitive is the method to this choice?
3. Have you investigated potential negative effects on downstream tasks?
1. Use out-of-distribution samples as keys is quite novel.
2. Provides stronger statistical guarantees than previous work.
3. Well-structured methodology presentation.
This paper introduces data taggants, a novel non-backdoor dataset ownership verification technique that helps detect if machine learning models were trained using a specific dataset. Unlike previous approaches that rely on backdoor watermarking, data taggants use pairs of out-of-distribution samples and random labels as secret keys, and employs clean-label targeted data poisoning to subtly alter a small portion (0.1%) of the dataset. When models are trained on the protected dataset, they respond to these key samples with corresponding key labels, allowing for statistical verification with only black-box access to the model. The authors validate their approach through comprehensive experiments on ImageNet1k using Vision Transformer and ResNet models, demonstrating that data taggants can reliably detect models trained on the protected dataset with high confidence, without compromising validation accuracy. The method proves to be stealthy, robust against various defense mechanisms, and effective across different model architectures and training recipes. It also provides stronger theoretical guarantees against false positives compared to previous approaches.
1. Lacks formal security analysis against adaptive attacks.
2. No investigation of downstream task impacts
Yes
No ethics review needed.
- Optimization is done only w.r.t. fully trained model parameters. Yet, the goal of the gradient matching is to make training a model from scratch on Taggants equivalent to training it on Keys. Why are some randomly initialized models not included? Do you have insight why despite this, the surrogate objective seems to work? 
- How should tau=0 on ln317 be interpreted? If I understand correctly, this means all models with non-zero accuracy on Keys are flagged?
- DOV is an important problem, and authors explicitly focus on realistic setups and often overlooked aspects such as the rigor of stated guarantees that accompany methods.
- The method is original within the space of DOV. The idea of repurposing witches brew and introducing random data sampling to strengthen the theoretical guarantees is very interesting and unexpected. 
- Evaluation focuses on a large-scale practical setup and addresses many important points, evaluating stealthiness and robustness explicitly. I appreciate the inclusion of poisoning defenses and OOD detection. 
- Setting the scope of evaluation aside (see below), the provided results seem quite strong.
- The paper is mostly well-written and easy to read, with some exceptions discussed below.
The paper proposes an active dataset ownership verification (DOV) method, by adapting a technique for targeted data poisoning from prior work. The key advantages of the proposed approach are its applicability given only top-k black-box access, more principled/rigorous statistical certificates compared to prior work, stealthiness, and robustness to different setups as well as explicit defenses. All these properties are validated via thorough experimental evaluation.
I can identify several important weaknesses of the work in its current state, and provide suggestions how these could be improved:
- **Incomplete evaluation/related work positioning**: While DOV is a crowded space and many baselines are cited in the paper, only two are run in the experimental part, without clear rationale, and the relationship to prior methods is in my view not clearly presented in the paper. For example, while the position of the paper seems to be "there may be DOV methods with strictly better TPR but they come with problems such as unrigorous guarantees or perceptible data changes", current Table 1 shows Taggants are the best even when only measuring TPR, which to me suggests that baselines are missing. The field is complex and there are many dimensions (active vs passive, blackbox top-k vs needs logits vs needs whitebox, different guarantee types, clean label vs perceptible, etc.). To give clarity, I believe the paper must (i) clearly outline all dimensions and place all prior baselines within them (ii) include any viable baseline (e.g., a perceptible method can be still run to demonstrate that even though it achieves high FPR, it fails a data poisoning defense) and clearly state why the others can not / should not be included. This would greatly improve the trust in the experimental results and make the case for Taggants.
- **Unclear claims of technical contribution**: The paper should clearly mark that many technical parts are directly lifted from Witches Brew (e.g., augmentations, restarts), while some other parts are introduced by this work (e.g., the use of random data, perceptual loss). The current writing can easily be interpreted as an overclaim, esp. by a reader not familiar with prior work. The actual contributions are quite interesting, and I do not think the lack of tech. contribution is a weakness of the paper in any case.
- **Unsubstantiated claims around guarantees**: One of the key claimed advantages of Taggants are rigorous guarantees not offered by prior work, as (i) random data samples are actually independent and (ii) under the null, the classifications of random data are actually uniform. While I tend to agree on an intuitive level, I believe (1) the reasons why prior work violates (i,ii) could be more clearly explained, e.g., ln301 simply states that "using model's predictions on ground truth class" violates the independence assumptions, but does not elaborate; (2) to show actual impact of this oversight of prior work, it should be empirically demonstrated that there is a mismatch between theoretical and empirical FPR (3) for taggants, there should be a corresponding matching FPR empirical validaiton, and a more detailed discussion around why taggants do not break the assumptions. Are model predictions on random [0,1]^d data really uniformly random? All these images are unusually high-variance compared to natural data; if we had a class such as "TV static" I can imagine they would all be classified as such? Do we need a different OOD distribution in this case, and how would we choose it? This needs more clarity as it is uite central to the paper.
- **[Minor] Key technical contribution undexplored**: If I understand correctly, the motivations given for how Keys are sampled are more rigorous guarantees as above, and lower likelihood to alter model utility, as data is OOD. However, Table 3 also shows forcing the model to predict a certain class is easier in this case than for in-distribution test images. If am not misinterpreting Table 3, it would be interesting to know why this is the case, and state it as the third reason for using such Key sampling to avoid confusion. Is it that gradient matching is here a better proxy for the true objective, or the objective is easier to optimize as we are far from the real data manifold? This seems underexplored but is a central idea of the paper. 

Typos and points that do not affect my evaluation:
- ln151: dot missing, ln518: extra dot, ln188: extra "them". ln317: "In each experiment..." sentence seems wrong, not sure where.
- Related work says "[Data/model] watermarks are not designed to persist through processes that use the data", but I am not sure this is really the case, as these watermarks are generally designed with the goal of robustness. There are works that show (albeit on text) explicitly that such watermarks can persist through processes of finetuning and RAG (see Sander et al. "Watermarking Makes Language Models Radioactive" and Jovanovic et al. "Ward: Provable RAG Dataset Inference via LLM Watermarks")---this discussion could be included to give context. On a similar note, the data/model/backdoor watermarks distinction could be made clearer, e.g. by changing the first paragraph title in Sec. 2.

I am happy to hear from authors regarding these points and discuss them further.
Yes
Yes, Privacy, security and safety
See the weakness part.
- The proposed method focuses on harmlessness, stealthiness, and black-box, which are three important challenges in the ownership verification problem.
- The writing is easy to understand and follow.
The paper proposed a dataset ownership verification method that can work in a black-box setting, where model weights and training details are not known in advance; Besides, the method is also stealthy compared to the backdoor-based method since it only requires limited perturbations to the dataset; Moreover, the method is also less harmful than the previous backdoor-based method.
- Novelty issues: Could you compare this paper with [1] in more detail, such as technical details, setting, and problem setup? Since in my understanding, [1] used a similar gradient-matching-based method to find some "hardly generalized domain", which is very similar to this method on a high level.
- Unclarified arguments: In Lines 59-60, the authors mentioned that 'but is also harmful to the model as it introduces errors [1]'. Could you further clarify what kind of errors the backdoor-based method introduces? In my personal understanding, the claim of "harmless" in [1] is mainly based on the fact that the backdoor-based method will leave exploits in the dataset, which will then further be maliciously used by the adversaries.
- Unclarified intuitions: The intuitions on why the "out-of-distribution" samples are used to construct key images are not further clarified.
- Experimental Details: Why do you choose SleeperAgent as the backdoor method for the baseline "Backdoor watermarking"? SleeperAgent is not the simplest way to inject backdoors and even requires an additional surrogate model to optimize perturbation $\delta$ to the original dataset. Therefore, could you (1) further clarify what is the necessity of choosing SleeperAgent, (2) provide more explanations on why the backdoor watermarking only achieves 0 TPR on your setting, and (3) provide additional experiments on the Backdoor watermarking with BadNet?

[1] Junfeng Guo, Yiming Li, Lixu Wang, Shu-Tao Xia, Heng Huang, Cong Liu, and Bo Li. Domain watermark: Effective and harmless dataset copyright protection is closed at hand.
Yes
N/A.
No ethics review needed.
1. The radioactive data (Sabrayrolles et al., 2020) method has the option of black-box verification. In black-box verification, the radioactive data method compares the difference in loss between clean and radioactive images, and it does not necessarily involve training a student model to replicate the suspected model, it just checks the difference between the loss. Thus, authors' claim in page 1, line 053 as well as in page 3 lines 127-129 are incorrect. I strongly recommend changing the explanation. 
2. I do not understand why the authors think that the independence of observations assumption does not hold in statistical testing. The models' predictions are independent of each other in the inference phase. 
3. The authors empirically show that the data taggants are visually imperceptible, as designed in the methodology. It can work quite nicely on images with a large input space, but my question is how imperceptible this noise will be in data with lower-dimensional input spaces, e.g., smaller images like CIFAR10, gray-scale images or on a different data type like tabular data or text? 
4.  In Table 1, the authors show that the backdoor watermarking (Li et al., 2023) has zero TPR and zero FPR. How authors measured such drastic numbers when the reference reports much better numbers? Is it because backdoor watermarking uses the full probability set instead of top-k labels or due to the mechanism of Wilcoxon-test? 
5. What happens if the adversary decides to use a subset of the dataset? It will negatively affect the verification as the ratio of signed images to the whole dataset might decrease. Another case is how the performance of Data Taggants is affected when the adversary combines different datasets to train its model? The budget B will decrease and smaller budget produce worse results according to Table 3. 
6. Page 2, line 148: typo while giving the reference
1. Empirical results show that Data Taggants have zero false-positive rate and high true positive rate while maintaining the model performance. 
2. The generation of secret keys is purely random and is not included in the modified dataset, which makes key recovery almost impossible and unique to the data owner. 
3. The presentation is clear.
In this paper, the authors propose Data Taggants, a dataset ownership method used to detect unauthorized data usage. Data Taggants relies on clean-label targeted data poisoning technique and requires only black-box access to the suspected model. Data Taggants generate secret keys, i.e., (input label) pairs, and signed input samples by maximizing the alignment between keys and signed samples, and induce a certain behavior only on the models trained by the modified version of the dataset including those signed images. The verification procedure of Data Taggants include statistical tests using suspected model's top-k predictions on the secret key.

I think there is novelty, particularly considering the application, but an incremental one as Data Taggants use ideas from gradient matching (Geiping et al. 2020).
1. As far as I understand, the verification includes querying the suspected model with keys. As they are purely random and out-of-distribution, the adversary might evade the verification by trying to detect those specific inputs and altering the predictions. 
2. The method might be prone to watermark collusion: the adversary can generate its own key set and data taggants by modifying the already signed dataset, and after that it can also claim that the accuser is the malicious one. 
3. Data Taggants has limited effectiveness and robustness when k=1 in top-k predictions.
We elaborate a harmless and stealthy targeted data poisoning approach to mark a model trained on a protected dataset and detect it.
@inproceedings{
anonymous2024data,
title={Data Taggants: Dataset Ownership Verification Via Harmless Targeted Data Poisoning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=6ldD8Y4gBQ},
note={under review}
}
Dataset ownership verification, the process of determining if a dataset is used in a model's training data, is necessary for detecting unauthorized data usage and data contamination.
Existing approaches, such as backdoor watermarking, rely on inducing a detectable behavior into the trained model on a part of the data distribution.
However, these approaches have limitations, as they can be harmful to the model's performances or require unpractical access to the model's internals.
Most importantly, previous approaches lack guarantee against false positives.\
This paper introduces _data taggants_, a novel non-backdoor dataset ownership verification technique.
Our method uses pairs of out-of-distribution samples and random labels as secret _keys_, and leverages clean-label targeted data poisoning to subtly alter a dataset, so that models trained on it respond to the key samples with the corresponding key labels.
The keys are built as to allow for statistical certificates with black-box access only to the model.\
We validate our approach through comprehensive and realistic experiments on ImageNet1k using ViT and ResNet models with state-of-the-art training recipes.
Our findings demonstrate that data taggants can reliably make models trained on the protected dataset detectable with high confidence, without compromising validation accuracy, and demonstrates superiority over backdoor watermarking.
Moreover, our method shows to be stealthy and robust against various defense mechanisms.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
dataset watermarking dataset ownership verification data poisoning backdoor attack
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/0b8dee7b6afadfa897b8ddddd01485ffbc522978.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Data Taggants: Dataset Ownership Verification Via Harmless Targeted Data Poisoning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission