TableTextGrad: A Reflexive Framework for Tabuar Understanding
Tabular Understanding;Table QA;Prompting
Yes
No ethics review needed.
Are there any plans to address the limitations of TableTextGrad in future work? For example, how to deal with large tables?
This paper adapts the TextGrad technique to the table understanding field, which leverages the Automatic Prompt Updating pipeline, which refines prompts through natural language feedback and gradient updates on training data. The contribution of this paper is simple and easy to understand.

From the experimental results, we observe that this paper demonstrates its ability to significantly improve performance in table question-answering tasks. Leveraging the “differentiation” of prompting pipelines refines each function within the Chain-of-Thought steps and function calls, leading to more accurate and reliable table reasoning outcomes. The results show that it achieves new state-of-the-art performance on benchmarks like WikiTableQA and TabFact, outperforming many existing methods.
The paper presents TableTextGrad, a framework for table understanding. Motivated by the recent advancements in the “textual gradient” space, this paper introduces TableTextGrad, a novel framework that enables automatic prompt optimization by leveraging the
“differentiation” of prompting pipelines through textual gradients. In the process, an initial LLM (Agent 1) generates table operations iteratively for table understanding. After each step, the table is updated based on the generated function calls and arguments. Then, in the validation phase, a second LLM agent (Agent 2) evaluates the predicted answers. If the answers are incorrect, natural language feedback on how to improve the prompt is backpropagated as textual gradients. These gradients are backpropagated to every prompting step used in generating the answer, including those for function selection, argument generation, and the final table query.  The framework is tested on datasets like WikiTableQA and TabFact and compared with various baseline methods.
Although this paper shows its merits in Dynamic Prompt Optimization and Soft Selection of Table Elements, it has some weaknesses that need improvement.

1. The novelty of borrowing the idea of TextGrad to the table understanding domain is limited since the contribution lies in the domain adaptation of the methodology, not belonging to the original contribution from scratch.

2. The performance of TableTextGrad decreases significantly when dealing with large tables. The increased complexity and token context required for large tables seem to lead to issues with memory and attention span within the model. If the authors have considered techniques like table chunking or hierarchical attention mechanisms to address the limitations with large tables.

3. The core reasoning capabilities of TableTextGrad rely on large language models like GPT - 4o and LLaMA 3.1, which are resource-intensive. Also, the improvements over these large models are slight and have not been evaluated with significant tests.
Yes
No ethics review needed.
1. Can this method be extended to more table tasks, such as text-to-SQL hybrid table question answering?
2. Can they explain the significant innovations that this method introduces based on textgrad?
1. The writing is relatively clear. 
2. The model has achieved good performance on two datasets.
The paper introduces TableTextGrad, a framework that enhances large language models' ability to reason over tables by automatically optimizing prompts through textual gradients. It combines the flexibility of inference-only techniques with data-driven learning, achieving state-of-the-art results on WikiTableQA and TabFact benchmarks. The approach refines prompts iteratively based on LLM feedback, improving accuracy in table reasoning tasks.
1. The evaluation datasets are relatively small, with tests conducted on only two datasets, and the evaluation tasks are quite limited. 

2. The authors claim to utilize the textgrad method.  It appears that tab textgrad is merely an application of textgrad.
Yes
No ethics review needed.
* In Figure 1, what is the right small rectangle in the "Gradient Update Example" part of the figure? What is its relationship with the left rectangle?

* In Sec. 4.2, you mention "the results" in line 396-397. However, I don't see it being referred to in the paper. Thus, in which table/figure are the results shown?

* You discussed training efficiency in Sec. 4.5. However, the discussion is abstract without empirical evidence. I'd like to see a comparison of running time & token consumption/cost against the baselines.
* Training-free methods are a useful approach to improving LLM performance without having to access their parameters. 

* Table understanding is an interesting and practical task. 

* The proposed technique makes intuitive sense, and shows strong performance.
This paper presents a prompt optimisation technique, named TableTextGrad, for the table QA understanding task. Specifically, TableTextGrad uses the *Chain of Table* method as the basis and applies the TextGrad idea to it to automatically optimises the chain of prompts used. 

Evaluation is done on two benchmark datasets: WikiTQ and TabFact, and TableTextGrad achieves the best performance, compared against a number of fine-tuning and inference-only methods.
* The proposed technique is incremental, and the novelty is a bit limited. It essentially applies TextGrad to the *Chain of Table* backbone. While effective, this is not exactly surprising that it works nor groundbreaking.
Yes
No ethics review needed.
See weakness
1. **Good Motivation for Automated Prompt Optimization**: The paper is well-motivated, highlighting the challenges of manual prompt engineering in table reasoning tasks and effectively positioning automated prompt optimization as a practical solution. The use of "textual gradients" for iterative refinement addresses the need for adaptive prompt tuning in large language models, making the approach both relevant and impactful.

2. **Comprehensive Ablation Study and Analysis**: The paper includes a thorough ablation study, providing valuable insights into the contributions of different components, such as soft versus hard selection and tuning all prompts versus only the final prompt. This detailed analysis strengthens the understanding of TableTextGrad’s performance and demonstrates the robustness and versatility of the proposed method across various model configurations and datasets.
This paper introduces *TableTextGrad*, a framework for enhancing table reasoning in LLMs through automated prompt optimization. It refines prompts iteratively using "textual gradients" based on model feedback, improving multi-step reasoning without extensive manual prompt engineering. Key innovations include extending Chain-of-Thought prompting with adaptive prompt adjustments and employing soft selection to retain broader table context, enabling more accurate comprehension in complex table tasks with minimal computational overhead.
1. **Questionable SOTA Claim**: The paper asserts achieving state-of-the-art performance; however, this claim is **not entirely accurate**. According to results in the E5 paper (Table 3) [1], E5 with GPT-4 achieves a score of 88.77 on TabFact, surpassing the 88.75 reported here. This discrepancy **raises concerns about the rigor of this paper's experimental claims** and the reliability of its benchmarking methodology.

2. **Limited Dataset Evaluation**: The evaluation is restricted to only two Table QA datasets (WikiTableQA and TabFact), which may not sufficiently demonstrate the generalization of the approach. Including a more complex and realistic dataset, such as HiTab [2], would provide a more robust assessment of the framework's applicability to diverse, real-world tabular data.

[1] E5: Zero-shot Hierarchical Table Analysis using Augmented LLMs via Explain, Extract, Execute, Exhibit and Extrapolate, NAACL 2024
[2] HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation, ACL 2022
"Textual Gradients" improves tabular understanding
@inproceedings{
anonymous2024tabletextgrad,
title={TableTextGrad: A Reflexive Framework for Tabuar Understanding},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=9mO9CNgNrh},
note={under review}
}
Table understanding is a complex task that requires not only grasping the semantics of free-form questions but also accurately reasoning over semi-structured tables. Recently, promising approaches designed sophisticated prompts that leverage large language models (LLMs) by combining Chain-of-Thought strategies with function calls, consequently demonstrating competitive results without requiring fine-tuning. 
However, creating sufficiently effective prompts remains a challenge. Without fine-tuning, all necessary priors must be incorporated directly into the initial prompt, making prompt design even more critical.
Motivated by the recent advancements in the ''textual gradient'' space, we introduce TableTextGrad, a novel framework that enables automatic prompt optimization by leveraging the ``differentiation'' of prompting pipelines through textual gradients. Concretely, according to the feedback of LLMs, TableTextGrad iteratively refines each function within the Chain-of-Thought steps and function calls, resulting in more accurate and reliable table reasoning outcomes. Experiments on table question-answering datasets demonstrate that our integrated approach achieves significant improvements, setting new state-of-the-art results on both the WikiTableQA and TabFact benchmarks. Our TableTextGrad not only enhances the reasoning capabilities of LLMs in the table reasoning task but also lays a groundwork for more robust and generalizable prompting pipelines due to its simplicity and effectiveness.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Tabular Understanding Table QA Prompting
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/001d4ac39f891d84a8f6f2008c542c1ac3e76994.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
TableTextGrad: A Reflexive Framework for Tabuar Understanding
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission