G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
Large Language Model;Mathematical Reasoning
Yes
No ethics review needed.
1. Will the RCU process generate a question with multiple answers, since sometime the cause and effect are not necessarily sufficient and necessary conditions? And unfortunately, the data correctness verification cannot recognize such case.
2. In Table 10, the performance gain led by cross-modal alignment seems limited, and why the fixed LLM performs better than tunable LLM?
- This paper curates a large-scale geometry dataset (28 times larger than the biggest existing geometry dataset) containing both detailed description of geometric images and reasoning paths, which may help MLLMs improve their ability in geometric comprehension.
- Data augmentation techniques like value scaling, equation solving, Re-Formulating conditions as unknown, and sentence paraphrase make it convenient to enlarge the size of dataset and enhance robustness.
- Extensive experiments validate G-LLaVA’s superiority in geometry problem solving. Experiments on MathVerse demonstrate its generalization ability.
Current Multi-Modal Language Models (MLLMs) struggle to solve geometry problems for lack of comprehension of geometry information. This paper generates a large-scale geometric problem dataset named Geo170K by GPT3.5, which contains more than 170K geometric image-caption and QA pairs. The data is made up of two components: the alignment data and instruction data. The former is generated through logical forms in Geometry3K while the latter augments QA pairs in GeoQA+. Besides, a model based on LLaVa is proposed to be trained on Geo170K. Numerical results show that G-LLaVA outperforms existing traditional methods and MLLMs like GPT4-V on the geometry problem solving (GPS) tasks.
- The experiments are all about choice questions. Some works also involve proving[1] and completion[2] questions, which can be more difficult. Experiments on such questions are neglected.
- In many experiments, the strongest existing model GPT4-V is not included, making the result less convincing.
- The proposed G-LLaVA selects LLAMA-2 as LLM and a pretrained ViT as the vision encoder. A comparative analysis or experiment on other architecture or pretrained model may help justify this choice.

[1] Chen J, Li T, Qin J, et al. Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression[J]. arXiv preprint arXiv:2212.02746, 2022.

[2] Li Z Z, Zhang M L, Yin F, et al. LANS: A layout-aware neural solver for plane geometry problem[J]. arXiv preprint arXiv:2311.16476, 2023.
Yes
No ethics review needed.
In Table 11, why does just including SP increase accuracy by 8%? This type of data augmentation doesn't enhance the model's understanding of geometry. Does this imply that the performance gains are due more to an improved understanding of the problem description rather than geometry?
1. Understanding geometric problems is important for MLLMs.
2. Presents a dataset with geometric alignment and instruction data.
This paper presents G-LLaVA, an MLLM focused on geometric problems. The authors propose that the difficulty in geometric problems lies in logical structure compactness. Subsequently, they introduce a new dataset, Geo170K, with geometric cross-modal alignment data and geometric instruction data to enhance the geometric perception ability of MLLM.
1. The novelty is limited, as it primarily fine-tunes an existing MLLM.
2. The models compared in Table 14 need to be updated; stronger baselines, such as QwenVL2, InternVL2 should be considered.
3. Since the authors claim that G-LLaVA enhances geometric understanding from image input, it would be interesting to see the G-LLaVA's  performance improvement when only input $Q$ is provided, in Table 8, 9 and 14.
Yes
No ethics review needed.
Please check the previous sections.
1. This work explores an interesting topic, geometric problem solving, and currently, most MLLMs perform poorly on this task.
2. The expression and organization of this manuscript make it easy for readers to understand.
3. The dataset proposed in this manuscript is relatively large and includes image-caption and question-answer pairs data.
This paper investigates the capabilities of existing MLLMs in geometric problem-solving tasks and introduces the Geo170K dataset, which contains over 170K image-caption and QA pairs. It also proposes training the G-Llava using the proposed dataset and validates its performance on the GeoQA dataset, demonstrating its effectiveness.
1. Novelty and Technical Contribution: The proposed geometric dataset construction approach relies on the existing logical annotation data and uses ChatGPT for automatic labeling. Besides, the overall model structure and training strategy mainly apply the existing techniques, lacking sufficient innovation. Please explain the differences in model structure or training strategy during the rebuttal phase and whether the authors made customized designs on a geometric domain. No specific considerations for the geometric domain were found.
2. Data Quality Issues: Since annotations are generated automatically by ChatGPT, is there any hallucination in the dataset? Is there any quality inspection conducted to support the claim of a "high-quality dataset"? 
3. Insufficient Evaluation: The paper lacks quantitative comparisons on geometry-task-related benchmarks like UniGeo, Geometry3K, and PGPS9K, commonly used in prior studies [1-4].

[1] Chen J, Li T, Qin J, et al. Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression[J]. arXiv preprint arXiv:2212.02746, 2022.
[2] Liang Z, Yang T, Zhang J, et al. Unimath: A foundational and multimodal mathematical reasoner[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023: 7126-7133.
[3] Zhang M L, Yin F, Liu C L. A multi-modal neural geometric solver with textual clauses parsed from diagram[J]. arXiv preprint arXiv:2302.11097, 2023.
[4] Li Z Z, Zhang M L, Yin F, et al. LANS: A layout-aware neural solver for plane geometry problem[J]. arXiv preprint arXiv:2311.16476, 2023.
Yes
none
No ethics review needed.
Please refer to the Weaknesses for more details. All in all, the authors are encouraged to append the discussion or experiments about recent progress in both dataset and methods.
It's a pioneering work on MLLM's mathematical reasoning especially focusing on geometry problem. Compared with its earlier version, the authors added more experiments on recent datasets like MathVerse.
The paper is a pioneering work about Geometry Problem Solving using MLLM. The contribution is to provide a dataset and also a baseline using LLaVA.
The work was a milestone in developing mathematical skills of MLLM and the authors tried to append results on a latest dataset. However, the research in the area moves fast in both dataset construction and methods. 
Despite the updates, the paper is still flawed in
1. Its targeted problem, i.e., geometry problem solving (GPS), is limited nowadays. The mathematical reasoning of MLLM evolves fast with the corresponding dataset. Now, it also covers the function plots in MathVerse and chart in MultiMath.
2. The baseline needs a fair comparison. Even though the problem is confined to GPS and methods in comparison is confined to open-source methods, the methods like MathLLaVA, MultiMath and Math-PUMA are encouraged to be compared with.
Generate mathematical geometry questions from various perspectives and fine-tune a powerful MLLM, advancing its capabilities in geometric problem-solving.
@inproceedings{
anonymous2024gllava,
title={G-{LL}a{VA}: Solving Geometric Problem with Multi-Modal Large Language Model},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=px1674Wp3C},
note={under review}
}
Large language models (LLMs) have shown remarkable proficiency in human-level reasoning and generation capabilities, which encourages extensive research on their application in mathematical problem solving. However, current work has been largely focused on text-based mathematical problems, with limited investigation in problems involving multi-modal geometric information. Addressing this gap, we aim to enable LLMs to solve geometric problems by understanding image input. We first identify the limitations of current Multimodal Large Language Models (MLLMs) in this area: they struggle to accurately comprehend basic geometric elements and their relationships. To address these challenges, we leverage the inherent attribute of logical structure compactness in geometric figures, utilizing text-only Large Language Models (LLMs) to curate a comprehensive multimodal geometry dataset. This dataset, named Geo170k, contains more than 170K geometric image-caption and question-answer pairs. Utilizing the Geo170k dataset, we introduce G-LLaVA, a model that demonstrates exceptional performance in solving geometric problems. It significantly outperforms GPT4-V on the geometry task of MathVista benchmark with only 7B parameters.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large Language Model Mathematical Reasoning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/492799138b06a879efac4ffabb0cd876378578b3.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/c0514e4753387256e5dfb495c2cda99d7f82d7ae.zip
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission