TabGraphs: A Benchmark and Strong Baselines for Learning on Graphs with Tabular Node Features
graph machine learning;tabular machine learning;graph neural network;gradient boosting;benchmark
Yes
No ethics review needed.
- Why does this paper not emphasize the high relevance of ML for relational databases? Why should relevant discussions go into the Appendix? The "Machine learning for graphs" paragraph in the related work took almost a page, but it is less relevant to ML for relational databases for this topic since the setting is essentially just a relational database with 2 tables.
- Why working with homogenous graphs a benefit, rather than a limitation? Extending homogeneous GNNs to heterogeneous GNNs only takes a few lines of code in PyG, and it makes the GNN model more expressive. I don't find homogeneous GNNs present benefits with heterogeneous GNNs.
- The experimental results are comprehensive. I appreciate the efforts from the authors to conduct experiments over 10+ baselines and 10+ datasets
- The paper cited a large number of related works, which helped the readers to know more about the field.
The paper introduces a benchmark to evaluate graph ML methods on tabular data with heterogeneous features. The paper hopes to bridge the gap between tabular and graph machine learning studies. It finds that while GNNs can enhance predictive performance, simple adaptations of standard tabular models could also work well.
- The problem setting is flawed. I do not agree that graphs with tabular node features are different from relational databases and can be redeemed as a new type of ML problem to work with. Indeed, the authors mentioned that "In contrast, our work is focused on single-table data with a single type of relationships between entities in the same table", and led the readers to Appendix C. But even after carefully reading the Appendix C, the paper still does not explain why the setting is different from a relational database with 2 tables (1 for the original table, the other for the additional relational information within the table). In that case, my understanding is that this paper is only a benchmark for relational databases with 2 tables, and only serves as a special case from the existing relational database benchmarks.
- Moreover, the authors did mention "Very recently, two benchmarks of large-scale relational databases have been proposed: RelBench (Fey et al., 2023) and 4DBInfer (Wang et al., 2024)." in the appendix. Given the high correlation with the proposed domain, they should be highlighted in the main paper and be thoroughly discussed, instead of hiding it within 1 sentence in the appendix. I'm not sure able the reason why the relevant discussions should not be mentioned in the main paper.
- The writing is a bit verbose. For example, the motivation in the introduction is weak. The first 2 paragraphs only introduce why we need tabular learning and graph learning, which is not relevant to the research problem in this paper.
- The paper makes little to none algorithmic contributions. Overall, I think this paper has not met the publication bar for ICLR.
Yes
No ethics review needed.
- More discussion is needed on the differences between this paper and the previous paper on this topic.

- Why there are models that suffer from performance drop after utilizing PLR?
- **Research Topic.** The studied problem is interesting and practically important for tabular machine learning.

- **Experiments.** Experiments are carried out on 10 datasets with 18 baseline methods, which is quite solid. And there are also enough discussion.
The paper explores enhancing predictive accuracy in tabular machine learning by incorporating graph structures, proposing benchmarks to bridge the gap between heterogeneous tabular and homogeneous graph data features. Results indicate that graph neural networks boost the accuracy of tabular predictions, and simple graph-based enhancements can enable traditional models to effectively compete. Valuable insights are provided for both tabular and graph machine learning practitioners.
- **Lack of Novelty.** The concept of constructing graph datasets from tabular data is not novel and has been previously suggested in multiple studies.

- **Writing.** The initial paragraph in the 'Machine Learning for Graphs' section of the related works is excessively lengthy, potentially hindering readability. Besides, The main text should provide more details about how tabular data is represented as graph data. The absence of these details in the main text may lead to confusion among readers.
Yes
No ethics review needed.
In the experiment section, the authors only compared the performance scores. I think it's also good to compare the runtime of the methods, as GBDTs can be much slower on larger datasets as compared to GNNs and tabular models.

The authors propose two types of tasks, node regression and classification. I'd like to see some other task types being covered, for example, multi-label classification, or link prediction. For example, in the hnm dataset, can you add a task to predict new items that will be co-purchases?

The authors should make proper comparisons to existing relational learning benchmarks RelBench and 4DBInfer.

Happy to raise the score if the authors can address all the three comments above.
Overall the writing is good and the presentation is clear. 
The experiments the authors provide are extensive, including different types of models, tabular dl models, GBDTs, GNNs, etc.
This paper proposes a benchmark for graph learning on single-table tasks(1 type of node and 1 type of edge), with heterogenous features. 
It constructs 11 tasks from 8 different datasets and ran experiments with GBDTs as well as GNNs on the tasks. 
The results show that GNNs outperforms GBDTs on the tasks authors proposed.
The novel contribution is limited in this paper.

On page 2, the authors claimed that GNNs are typically evaluated on homogeneous features, but tabular learning typically has heterogenous features. I think they omit the work of PyTorch Frame (Hu, Weihua, et al. "PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learning." arXiv preprint arXiv:2404.00776 (2024). ), which supports encoding heterogenous tabular data through deep encoders and can be used with PyG on downstream GNN tasks. It is good to include comparison to PyTorch Frame + PyG in your paper as it also fits into graph + heterogenous tabular data. 

The authors claim on page 4. "In contrast, our work is focused on single-table data with a single type of relationships
between entities in the same table.". However, it seems to me all the graph structure constructed by the authors can be represented by the graph structure in Relbench as well.

I also think the task type is too limited. Currently it only supports node classification and node regression. The authors did not mention link prediction tasks. Link prediction is a common tasks solved by GNNs and is offered by Relbench. It is unfair to say node level task is "the most common setting in modern graph machine learning".
We address the gap between tabular and graph machine learning by collecting a new benchmark of meaningful tabular datasets with known graph structure and introducing strong and simple baselines previously overlooked by research community.
@inproceedings{
anonymous2024tabgraphs,
title={TabGraphs: A Benchmark and Strong Baselines for Learning on Graphs with Tabular Node Features},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=a6XE2GJHjk},
note={under review}
}
Tabular machine learning is an important field for industry and science. In this field, table rows are typically treated as independent data samples, but additional information about the relations between these samples is sometimes available and can be used to improve predictive performance. Such information can be naturally modeled with a graph, hence tabular machine learning may benefit from graph machine learning methods. However, graph machine learning models are typically evaluated on datasets with homogeneous, most often text-based node features, which are very different from heterogeneous mixtures of numerical and categorical features present in tabular datasets. Thus, there is a critical difference between the data used in tabular and graph machine learning studies, which does not allow one to understand how successfully graph models can be transferred to tabular data. To bridge this gap, we propose a new benchmark of diverse graphs with heterogeneous tabular node features and realistic prediction tasks. We use this benchmark to evaluate a vast set of models, including simple methods previously overlooked in the literature. Our experiments show that graph neural networks indeed can often bring gains in predictive performance for tabular data, but standard tabular models can also be adapted to work with graph data by using simple graph-based feature augmentation, which sometimes enables them to compete with and even outperform graph neural models. Based on our empirical study, we provide insights for researchers and practitioners in both tabular and graph machine learning fields.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
graph machine learning tabular machine learning graph neural network gradient boosting benchmark
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/509c1e08d1f893330c5e9b4885b5c844fed868db.pdf
datasets and benchmarks
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
TabGraphs: A Benchmark and Strong Baselines for Learning on Graphs with Tabular Node Features
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission