Improving Defense Mechanisms for Subgraph-Structure Membership Inference Attacks
Graph neural network;subgraph-substructure membership inference attack;2-stage training
Yes
No ethics review needed.
1. The problem formulation is not very clear. If I have the adjacency matrix of the graph available, why do I need to create a model to predict if the nodes form a particular structure or not? What setup is used for the experiments here - transductive or inductive?

2. Why small scale datasets are used for the SMIA attack? More statistical information about the dataset (structure you are trying to find) should be included to show the motivation of the work.

3. What kind of adaptive attack can be developed against the proposed solution?
1. The paper is well written.
2. Substantial experimental results are included in the paper.
In this paper, the authors explore the robustness of graph neural networks (GNNs) against membership inference attack (MIA), specifically subgraph-structure membership inference attack (SMIA) where the foal is to infer whether a set of nodes form a particular structure e.g., a clique or multi-hop path in the training graph. The authors propose an SMIA attack which uses multiset function to improve the impact of the attack. To combat the attack, they propose a two-stage defense strategy. In the first stage, the model is trained on the training set, with some one-hot labels converted to soft labels to add ambiguity to selected samples. In the second stage, this model is further refined on the test set, where pseudo-labels are generated based on the model’s predictions after the first stage of training. Experimental results are included to support the claims.
1. The methodology section of the paper is lacking. The authors include comparison with other attacks and defenses in the methodology section which is not recommended. This section of the paper should emphasize on the proposed solution and the novelty. It is not clear as to why the new attack is more effective. Some explanations should be provided.
 
2. Novelty of the paper is lacking. Flattening is used by other defenses too [1].

3. For the proposed defense solution, the authors predict the pseudo-labels for the test nodes which is then used for training the model further. What happens when we have a dynamic graph where new nodes are added? Will the defense be still effective?

4. No adaptive attack is developed against the proposed defense solution to show its effectiveness. 

[1] RelaxLoss: Defending Membership Inference Attacks without Losing Utility
Yes
No ethics review needed.
Please refer to weaknesses.
1. Sufficient empirical comparisons. Authors present lots of comparisons with other defense baselines to demonstrate advantages of TSD.
2. Clear writing and easy-to-follow.
This paper proposes a defense strategy against SMIA attacks. To defend sub-graph and label inference from GNNs, this paper devises a two-stage defense method (TSD) consisting of a flattening strategy and test set adjustment strategy.  The flatten strategy aims to introduce noises to the label distribution and the test set adjustment will reduce differences in outputs between the training set and the test set.
1. Significant performance deterioration. Especially in Table3, after TSD defense, clean performance deteriorates from 80\% to 77\%. Performance deterioration problem is a significant problem in studies of defending against MIA attacks. I suggest authors also focus on tackling this problem.
2. Unclear motivation. The two-stage method is described clearly, but the motivation is unclear. For instance, Flatten strategy aims to introduce noises to the label distribution, but why could this help defend SMIA? In the second stage, does the test set adjustment aim to reduce differences in outputs between the training set and test set to defend against SMIA? If so, almost all defense motivation follows this motivation. Hence, I suggest more description or demonstration of motivations.
Yes
No ethics review needed.
*  For SMIA, the paper assumes that the attacker can access sufficient posterior distributions for an effective attack. However, it limits the range of application of this paper as that attackers might not be always available for obtaining such posterior information.

* In the defense, TSD uses soft labels to change the loss distribution, assuming that this label smoothing works well in all situations. However, for some tasks, using pseudolabels instead of actual labels might lower the model’s performance or cause unexpected biases.

* TSD assumes that it can remain effective across various types of GNNs and datasets. I am curious whether TSD can maintain the same level of performance on heterogeneous (not heterophilic) graphs is questionable.
- The paper is well written and technically sound. 
- The attention on defenses on SMIA should be valued.
- The experiment are conducted thoroughly.
This paper discusses subgraph structure membership inference attacks (SMIA) on GNNs, which is an important yet unattended problem, and presents a novel defense mechanism. Specifically, for the attack method, the paper introduces an end-to-end SMIA attack model based on multiset functions, which seems to be a powerful attacker due to the order-invariant property. For the defense method, the paper designs a two-stage training strategy and a flattening technique to alter the posterior distributions and enhance defense capabilities.
I have the following concerns about the settings of attacker and the effectiveness of the defense methods.
Yes
No ethics review needed.
The paper proposed novel defenses and attacks on the domain of subgraph-structure membership inference attacks against graph neural networks. The authors first introduce the limitations of the previous state-of-the-art attacks. Then, the authors introduced the novel defense methods to defend against the attacks. Finally, the authors also improved the attacks. Experiences are conducted to demonstrate the effectiveness of the proposed defenses and attacks. The paper is well-written. However, I have several comments regarding motivation and performance.

- The authors proposed both defenses and attacks. The defenses can also mitigate the proposed new attacks. Therefore, I am wondering what is the motivation of the proposed attacks. The authors are supposed to explain why the proposed new attacks are important and what conclusions can be achieved by the new attacks.
- The defense method does not work that well. The attack success rate is still high enough to pose a threat. Therefore, the authors are suggested to discuss why the attacks can still achieve moderate performance.
- Instead of introducing new attacks, I recommend that the authors consider adaptive attacks against the proposed defenses. i.e., what can the adversary do if they know the defense methods?
- Trendy topic
- Well-written paper
The paper focuses on the defense strategies for graph neural networks against subgraph-structure membership inference attacks (SMIAs).Specically, the paper defend against the state-of-the-art SMIAs using flattening. The authors also strengthen the current attacks. Extensive results demonstrate that the proposed defense method can decrease the attack performance.
- Unclear motivation
- Performance is not that good
- Lack of adaptive attacks
@inproceedings{
anonymous2024improving,
title={Improving Defense Mechanisms for Subgraph-Structure Membership Inference Attacks},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=e5g53a4A0g},
note={under review}
}
Graph neural networks (GNNs) are of significant importance in diverse real-world applications since they leverage powerful graph learning techniques to solve problems pertaining to social network mining and medical data analysis. Despite their practical relevance, GNNs remain vulnerable to adversarial attacks such as membership inference attacks (MIAs) which pose privacy risks by revealing whether specific data records were part of the training set of the model. While most existing research has focused on designing defense mechanisms for known node-level MIAs, and in particular, for determining if a certain node was used during training, only limited attention has been paid to subgraph-structure MIA (SMIA) problems. SMIA methods seek to infer whether a set of nodes forms a particular target structure of interest (such as a graph motif, e.g., clique or multi-hop path) in the training graph. The main contributions of our work are three-fold. The first is a novel robust defense mechanism for GNNs against SMIA attacks. It  combines an alternating train-test schedule with a flattening strategy to mitigate the attacks. The second contribution is a new end-to-end SMIA attack model that outperforms existing attacks by using multiset functions to generate learnable embeddings for collections of nodes. Extensive simulations reveal that the new attack model outperforms prior state-of-the-art attack models on GNNs by 12.31% across four datasets when no defense mechanism is present. With the new defense mechanism, one can achieve an average decrease of 14.30% in the attack AUROC and an 10.05% improvement in target model utility compared to classical defenses, even when using the improved attack scheme. The third contribution is a study that shows that our defense mechanism extends to node-level MIAs as well, offering similar improvements in attack resistance and utility.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Graph neural network subgraph-substructure membership inference attack 2-stage training
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/7df77a2d363255768286dc08810b81a73f3f45b0.pdf
learning on graphs and other geometries & topologies
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/85e370f1a4197b741bb6c14af709fb9b82b39f2d.zip
Improving Defense Mechanisms for Subgraph-Structure Membership Inference Attacks
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission