BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models
Large language models;Reasoning;Planning;Trustworthiness;Interpretability;Probability Estimation;Bayesian Methods
Yes
No ethics review needed.
In Section 4.3, "The Usage of the Reliably Estimated Probability", it seems that the original T5 large model is better on some datasets compared to the fine-tuned models. What would be the reasons for this? Besides, what is "PatternTime" in Table 3? It is not mentioned anywhere else in the manuscript.

Please see "Weaknesses" for a few additional questions.
1. The proposed Bayesian inference framework is conceptually interesting. Factoring the overall decision-making into multiple fine-grained factors is intuitive and useful for mitigating LLMs' limitations in providing accurate fine-grained probability/confidence estimation.

2. The experiments performed are carefully designed, and the overall writing is clear and with enough details.

3. The experiment results demonstrate the effectiveness of BIRD, showing its value in multiple application scenarios.
This work proposes an inference framework that aims to produce more reliable and accurate probability/confidence estimations using LLMs for large-scale decision-making and planning tasks. The proposed method, BIRD, uses abductive factor generation to factorize the inference task, and estimates the outcome by accumulating the predictions made on the specific factors. The experimental results on three datasets show that BIRD is able to produce more accurate probability estimations in decision-making and planning, which can be used to (1) improve the accuracy of decision-making; (2) supervise the training of small models; (3) propose follow-up questions for decision-making. For all these applications, BIRD shows superior performance than the baseline methods.
1. The training setting for "Learning algorithm for constrained optimization to estimate $P(O_i|f)$" (Line 266) should be more clearly described. The current description does not clarify (1) the number of learnable parameters ($P(O_i|f_j)$); (2) whether different instances share the same factors $f_j$; (3) whether there are $f_j$ in the test data that are unseen in the training data.

2. In the experiment for "Applying BIRD’s Probabilities in Decision-Making" (Line 430), the test instances on which BIRD's decision is unknown are filtered out. While it is claimed that "this does not undermine our experiment setting because such removal is label-agnostic" (Line 450), it may still introduce biases since this filtering process depends on the output of BIRD. It might be the case that BIRD predicts "unknown" when it is uncertain internally.
Yes
No ethics review needed.
There several magic numbers in 3.3 and 3.4 (mappings of rankings to probabilities), as well as key design decisions in model formulation. Can you comment on sensitivity of the method to these values/decisions?

Will the evaluation data - and the code - be released?

372, 449: What instances lead to 'unknown' predictions for all factors? Is this a shortcoming with the factor generation method, or is it that the problem itself is riddled with uncertainty.

452: Can you elaborate on the spurious biases that affect CoT, giving it such strong performance? The same criticism could be applied to the LLM prompting as part of Bird, and the probabilities produced.
This paper presented a very novel idea, a detailed method for achieving this, and a convincing evaluation of the utility of the technique. Specific strengths include:
 - mechanism for prompting a LLM to extract factors & potential outcomes of a complex decision problem
 - means to convert the factors into a binary Bayes network
 - incorporation of model uncertainty estimates into the parameterisation of the Bayes network
 - evaluation against human judgements of probability
 - strong baselines
This paper presents a method allowing LLMs to reason more soundly about complex problems with uncertainty, by combining LLM introspection and verbal estimates of uncertainty into Bayesian reasoning about outcomes. In many ways it resembles chain-of-thought, in that the LLM is being asked to provide high level reasoning about an uncertain scenario, and is required to brainstorm to solve the problem. The incorporation of a Bayes net sets the method apart from prior work, and experiments show that this leads to much better estimates in pairwise evaluation of propositions, but less compelling evidence for the work when evaluated on decision making benchmarks.
The paper was quite dense to start, and took several pages to make the application scenario clear. On my first pass it wasn't apparent to me what elements of the data were part of the problem definition, what parts were computed by the LLM, and what was computed by inference in a Bayes net. Fig 1 is reasonably easy to follow, but Fig 2 is less clear - why have the conditions changed between Fig 1 and Fig2, and what parts of the central box are given as part of the data versus inferred from LLM interactions? As I understand it, the whole of the inner box is auto-generated, as well as the edges linking the given conditions with coarse probability estimates.  

The paper also takes a long and detailed dive into Bayes net formality, which is more than needed given the setting is relatively simple and could be communicated effectively with more of a focus on the running example (Fig 2 could be unpacked). As it stands, the paper takes several pages to get going, but ends up deferring many critical details to the Appendix. The method for prompting the LLM is of fundamental importance to the technique, for instance, but none of this is in the main paper.
Yes
No ethics review needed.
1) Page 1: LLMs’s → LLMs’
2) The LaTeX expression for P in the introduction differs from its expression in the later sections.
3) Where is the example of the factor, "The color of the vehicle," mentioned by the authors in Line 228, depicted in Figure 2? I couldn't find it in the figure as described in the text.
4) Appendix A.2: Week →Weak
5) This may be a minor issue, but there could be confusion between Line 318 'S + U₃ implies d₁ and f₂' and f, which is a possible instance of complete information due to the notation.
6) An additional question is, have the authors considered any experiments involving more than two outcomes (i.e., beyond binary classification)?
1) The proposed framework and optimization settings, which account for weak ordering and non-interaction in the set of factors, are valid.
2) The authors' assumptions about enhancing the validity of the proposed Bayesian Inference Framework are reasonable.
3) Including unobserved factors in the calculations strengthens the effectiveness and utility of the authors' proposed method.
4) Besides precise probability estimation, the presence of follow-up generation and cross-domain experiments demonstrates the proposed method's reliability and practicality.
This paper proposes a BIRD framework that enables more reliable probability estimation based on incomplete instances from LLMs. It leverages factors generated by LLMs to make a Bayesian network and estimates more precise probabilities. BIRD with optimized prob presents a better performance in several experiments than existing methods. BIRD also demonstrates improved reasoning performance across various datasets, particularly excelling in more complex scenarios.
1) In Line 147 and Line 155, the explanation of the outcomes is separated. It seems possible to consolidate these explanations into one part. Additionally, the explanation of 𝐹 in the same paragraph could be made more concise. The space saved from these revisions could then be used to more clearly elaborate on the connection between Equation (5) and Equation (6) and the content in Algorithm 1.

2) Ablation settings could be expanded to cover more aspects. For example, experiments could be conducted by excluding the MR loss from the unified loss function or by assuming that in an unobserved factor, each value does not have an equal probability of being selected.
@inproceedings{
anonymous2024bird,
title={{BIRD}: A Trustworthy Bayesian Inference Framework for Large Language Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=fAAaT826Vv},
note={under review}
}
Predictive models often need to work with incomplete information in real-world tasks. Consequently, they must provide reliable probability or confidence estimation, especially in large-scale decision making and planning tasks. Current large language models (LLM) are insufficient for such accurate estimations, but they can generate relevant factors that may affect the probabilities, produce coarse-grained probabilities when the information is more complete, and help determine which factors are relevant to specific downstream contexts. In this paper, we make use of these capabilities of LLMs to provide a significantly more accurate probabilistic estimation. We propose BIRD, a novel probabilistic inference framework that aligns a Bayesian network with LLM abductions and then estimates more accurate probabilities in a deduction step. We show BIRD provides reliable probability estimations that are 30% better than those provided directly by LLM baselines. These estimates can further contribute to better and more trustworthy decision-making.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large language models Reasoning Planning Trustworthiness Interpretability Probability Estimation Bayesian Methods
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/61bb3e15168ce2b9fa8905abeb5b613d4c31cce3.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission