Centrality-guided Pre-training for Graph
Graph neural networks;Graph representation;Graph pre-training;Graph centrality
Yes
No ethics review needed.
1. Why do you choose $L_2$ as your alignment loss function not a distribution alignment loss function like KL-divergence?
1. It proposes a novel self-supervised framework for pre-training the graph that considers both structure embeddings and graph embedding based on node features only.
2. it introduces a novel alignment module with a loss function that aligns the structure embeddings from a global perspective and node embeddings based on node features only.
3. the experimental results outperform baselines significantly.
The paper introduces a novel self-supervised framework for graphs, which considers aligning graphs' representation and structure based on centrality in graph theory. It contains three modules of node representation pre-training and alignment. The node-level structure learning aims to enhance the node representations based on degree centrality to assign similar embeddings for nodes with the same degree. The graph-level structure learning is based on eigenvector centrality to provide information on node importance from a global point of view. Specifically, it uses the graph representation to guide the structure matrix in determining node importance through a cross-attention module. at last, a graph representation alignment module is applied to minimize the discrepancy between the pre-trained structural embedding and embeddings learned by other graph models. The experimental results demonstrate the effectiveness of the framework.
1. The use of SVD limits the applicable ability from very large graphs.
2. based on Table 4, the proposed framework is most effective for sparse graphs. For relatively dense graphs like Photo, the improvement is limited.
3. combined with Tables 4, 5, and 6. the proposed framework is most effective for graph classification while it has limited improvement for node classification and edge classification. 
4. some writing errors, e.g., missing "is" in "which the maximum absolute" at line 267, capital "D"at line 245
Yes
No ethics review needed.
1、The paper employs degree centrality and eigenvector centrality to measure node importance. What are the specific reasons for choosing these two centrality metrics? Were other centrality measures considered, and if so, why were they not selected? It is recommended to explore a broader range of centrality metrics and analyze their suitability for different types of graph data and tasks. Experimental comparisons of the CenPre framework's performance under different centrality metrics could further optimize how node importance is measured.

2、The CenPre framework consists of three modules. How do these modules interact with each other? For instance, when integrating information between the node-level and graph-level structural learning modules, is there interference or synergistic enhancement? How could experimental or theoretical analysis be used to understand the interactions between these modules more deeply? A more detailed experimental design, examining the impact of various module combinations on model performance, would allow for an in-depth analysis of module synergy. From a theoretical perspective, a mathematical model describing the information flow and interactions between modules could provide a foundation for further optimizing the framework structure.

3、In the experimental section, for hyperparameter settings such as λ1, λ2, and λ3 in the balanced loss component, it is mentioned that these values were determined through pilot studies, but the specific tuning process and rationale were not elaborated. Could you provide more details on the hyperparameter selection process to help readers better understand the model’s sensitivity and stability?
1、The authors propose a novel centrality-guided graph pretraining framework, CenPre, which integrates graph structural information with node representations. This systematic application of the centrality concept in graph pretraining is innovative.

2、Unlike traditional graph representation learning methods that rely solely on augmentation, this paper introduces an innovative approach by learning node importance at both the node and graph levels, with corresponding modules designed for structural information integration.

3、The experimental section conducts a comprehensive evaluation on 13 benchmark datasets across various domains and scales, covering node classification, link prediction, and graph classification tasks. The rigorous experimental setup and comparisons with numerous advanced baseline models enhance the reliability and persuasiveness of the results.
In this paper, the authors propose the CenPre framework, which aligns graph structure information with representations based on the concept of centrality. By learning importance at both the node and graph levels, CenPre enhances node representation quality through pretraining and alignment modules.

The main contributions include:

Aligning graph representations with structural information during pretraining to generate improved graph representations for downstream tasks.
Introducing the CenPre framework based on centrality, capable of learning structure-integrated graph representations from both local and global perspectives.
Demonstrating, through experiments on multiple real-world datasets, that CenPre significantly outperforms baseline models in node classification, link prediction, and graph classification tasks.
1、The choice of centrality metrics by the authors is relatively traditional, lacking exploration of alternative, potentially more suitable centrality measures for complex graph structures or specific application scenarios.

2、There is an absence of sensitivity analysis on the model under different parameter settings, making it difficult for readers to understand the stability and performance trends of the model with various hyperparameter values.

3、The theoretical explanation of the relationship between centrality, graph structure, and node representation lacks depth, missing a more rigorous mathematical basis to explain why this method can effectively improve graph representation learning performance.

4、The analysis of computational and spatial complexity is not comprehensive, with a lack of detailed efficiency comparisons to existing methods.
Yes
No ethics review needed.
same as cons
- The author proposed a new method CenPre framework enhances graph representations by integrating structural information, specifically node importance via centrality measures, into self-supervised learning. 
- Experiments show that CenPre outperforms current methods in node classification, link prediction, and graph classification.
Summary:
The author proposed CenPre framework enhances graph representations by integrating structural information, specifically node importance via centrality measures, into self-supervised learning. CenPre includes modules for node-level structure learning using degree centrality, graph-level structure learning with eigenvector centrality, and a representation alignment module that retains semantic integrity while incorporating structural insights. Experiments show that CenPre outperforms current methods in node classification, link prediction, and graph classification.
- More self-supervised learning methods with different training paradigms are encouraged to be added. GBT[1] using barlow twins and GGD[2] using group discrimination.
- There are three hyperparameters λ1/2/3 to be tuned. It seems it is quite hard to tune these parameters.
- Has the pretraining methods tried to employed to be pretrained on a set of other datasets first and then evaluate on a target dataset? It seems the method is still dataset-specific? The L_d and L_e only relies on graph topology and seems to be able to used in this setting.


[1] Bielak, P., Kajdanowicz, T., & Chawla, N. V. (2022). Graph barlow twins: A self-supervised representation learning framework for graphs. Knowledge-Based Systems, 256, 109631.
[2] Zheng, Y., Pan, S., Lee, V., Zheng, Y., & Yu, P. S. (2022). Rethinking and scaling up graph contrastive learning: An extremely efficient approach with group discrimination. Advances in Neural Information Processing Systems, 35, 10809-10820.
Yes
No ethics review needed.
1. Can the authors provide more evidence to support their basic opinion?
2. Can the authors make further discussion on the differences with existing works?
3. The third module aligns the centrality-guided node representation with "the original pre-trained model (line 320)." How is the "original pre-trained model" obtained? Does this mean in CenPre we need to conduct pre-training twice? The overall workflow of the method needs a clearer elucidation.
4. What does "CenPre uses the original node representation" mean in line 484?
1. The experiments are comprehensive. A large number of datasets from various tasks are included, validating the effectiveness of the proposed method.
2. This paper is easy to read.
To address the problem that existing graph SSL methods focus on augmentation but overlook the integration of structural information, this paper proposes a Centrality-guided Graph Pre-training (CenPre) framework to learn better graph representations that incorporate structural information. CenPre consists of three modules that use degree, eigenvector centrality, and original graph representations to conduct aligning during pre-training. Experimental results demonstrate the effectiveness of CenPre on various tasks and datasets.
1. **The motivation is not well explained.**
    
    As the foundation of this paper, the authors claim that existing graph SSL methods only focus on augmentation and neglect the integration of structural information. This is somewhat not convincing. a) It is well known that GNNs can effectively capture both semantic and structural information simultaneously, especially considering some GNN variants with stronger expressive power. It is arbitrary to say that existing SSL methods fail to incorporate structural information because they can implicitly achieve this. b) At the same time, it is also common practice to explicitly consider structural information in existing SSL works. More theoretical or experimental evidence should be provided regarding the limitations of existing methods.
    
2. **The method lacks novelty.**
    
    The proposed CenPre basically uses centrality to guide the representation learning during pre-training. However, the concept of centrality has already been widely introduced in GNNs [1,2,3,4,5]. Among them, [4,5] exactly use centrality as a pretext task for pre-training. A detailed discussion on the differences between CenPre and previous works is necessary, which is missing in this paper.
    

Minor Point:

The paper uses a considerable amount of space (Sec 4.1 and Sec 4.2) to introduce the well-known concept of centrality, which is unnecessary.

[1] Maurya, Sunil Kumar, Xin Liu, and Tsuyoshi Murata. "Fast approximations of betweenness centrality with graph neural networks." *Proceedings of the 28th ACM international conference on information and knowledge management*. 2019.

[2] Avelar, Pedro, et al. "Multitask learning on graph neural networks: Learning multiple graph centrality measures with a unified network." *International Conference on Artificial Neural Networks*. Cham: Springer International Publishing, 2019.

[3] Zhu, Yanqiao, et al. "Graph contrastive learning with adaptive augmentation." *Proceedings of the web conference 2021*. 2021.

[4] Jin, Wei, et al. "Self-supervised learning on graphs: Deep insights and new direction." *arXiv preprint arXiv:2006.10141* (2020).

[5] Hu, Ziniu, et al. "Pre-training graph neural networks for generic structural feature extraction." *arXiv preprint arXiv:1905.13728* (2019).
We propose CenPre, a framework that integrates structural information into node representations using centrality measures, outperforming state-of-the-art methods in node classification, link prediction, and graph classification.
@inproceedings{
anonymous2024centralityguided,
title={Centrality-guided Pre-training for Graph},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=X8E65IxA73},
note={under review}
}
Self-supervised learning has shown great potential in learning generalizable representations for graph-structured data. However, existing methods largely focus on improving graph representations based on augmentations, which ignores the alignment between the representation and the structure of graphs. To fill this gap, we propose a Centrality-guided Graph Pre-training (CenPre) framework to integrate the structure information into the representations of nodes based on the centrality in graph theory. The proposed CenPre contains three modules for node representation pre-training and alignment. The node-level structure learning module fuses the fine-grained node importance into node representation based on degree centrality, allowing the aggregation of node representations with equal/similar importance. The graph-level structure learning module characterizes the importance between all nodes in the graph based on eigenvector centrality, enabling the exploitation of graph-level structure similarities/differences when learning node representation. Finally, a representation alignment module aligns the pre-trained node representation using the original one, essentially allowing graph representations to learn structural information without losing their original semantic information, thereby leading to better graph representations. Extensive experiments on a series of real-world datasets demonstrate that the proposed CenPre outperforms the state-of-the-art baselines in node classification and achieves better performance in link prediction and graph classification than the baseline models.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Graph neural networks Graph representation Graph pre-training Graph centrality
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/8eabb2233fcddb00c7047eb95b28e64f06611fd7.pdf
learning on graphs and other geometries & topologies
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Centrality-guided Pre-training for Graph
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission