Unsolvable Problem Detection: Evaluating Trustworthiness of Large Multimodal Models
Unsolvable Problem Detection: Evaluating Trustworthiness of Large Multimodal Models
Yes
No ethics review needed.
1. Please refer to the concerns raised in Weaknesses 2 and 4.

2. In addition to the three challenges defined in the paper, have the author considered an additional challenge of *incompatible question detection*? This would involve scenarios where the image and answer set align, but the question itself is incompatible with both the image and the answer. For example, replacing the original question in an image-question-answer set with an unrelated one. This condition appears aligned with how the proposed challenges are structured. Would it be necessary to incorporate this condition, or is this concept already encompassed by the current challenges defined?
1. The paper is well-written and easy to follow. The definitions of the problems and evaluation criteria are clearly presented, and the visualizations of examples and results enhance the readers' understanding.
2. The paper conducts comprehensive and detailed experiments to analyze where and how LMMs struggle with UPD. It further evaluates the effectiveness of existing methods in addressing UPD, which offers valuable insights for future researchers aiming to tackle these challenges.
The paper introduces Unsolvable Problem Detection (UPD) as a framework to evaluate Large Multimodal Models (LMMs) on their capacity to identify and manage unsolvable problems. This framework consists of three main challenges: Absent Answer Detection (AAD), Incompatible Answer Set Detection (IASD), and Incompatible Visual Question Detection (IVQD), to address scenarios involving insufficient, mismatched, or incompatible information. To support evaluation, the authors present the MM-UPD Bench, a benchmark consisting of 2k questions derived as a subset of MMBench, to measure LMM performance in these areas. The paper evaluates several LMMs using the MM-UPD Bench and provides an analysis based on open-source versus closed-source models, as well as detailed question types.

The experimental results reveal that even state-of-the-art LMMs, which perform well on conventional benchmarks, encounter difficulties when assessed with the MM-UPD Bench. Additionally, generic existing approaches were tested to address UPD but proved ineffective, which underscores the challenge of UPD.
1. The paper's strategy for filtering image-agnostic questions (line 207) to ensure that problems truly rely on the image modality seems problematic. Specifically, the method filters questions from MMBench that can be answered correctly using text-only GPT-4 and manually reviews the remaining questions. This approach risks excluding genuinely image-dependent questions that GPT-4 might answer correctly by chance. As shown in Table 1 IVQD, there is a possibility that GPT based model attempts to answer questions that require image information but lack relevant image input. This could lead to false exclusions, and introduce bias into the dataset and impact the evaluation of GPT-based models.
2. Following the previous point, the paper states in line 996 that 13% of the original questions are removed as image-agnostic. This suggests that the initial GPT-based filtering only marginally reduces manual effort. Given this, why not manually filter all image-agnostic questions to ensure greater accuracy and consistency?
3. The use of Dual Accuracy as the primary metric (line 252) may not fully capture the intended evaluation properties of the benchmark. Dual Accuracy is defined as *the accuracy on standard-UPD pairs, where success is counted only if the model answers both the standard and UPD questions correctly.* However, as illustrated in Figure 2's examples, a model might correctly refuse to answer absurd questions even if it is not capable of answering the standard version. This coupling of general VQA ability and the capacity to reject unanswerable questions could skew comparisons across models. To better assess the models' capability in detecting unsolvable questions, it may be more informative to compare success rates on questions where the model correctly answers the standard version. This ensures both text and image are understood, and evaluates if the changes that make the question becomes unanswerable are detected. (Mathematically, it's similar to divide Dual Accuracy by Standard Accuracy). 
4. The distribution of question types appears skewed and inconsistent across AAD, IASD, and IVQD, as shown in Table B (line 1042). For instance, in AAD and IASD, the three most common question types (#2, #17, and #3) account for 30% of the questions, whereas in IVQD, the most common types (#2, #7, and #9) account for 44%. Could this affect the analysis of UPD performance? Additionally, six question types have no representation in IVQD. While it is understandable that certain types need to be removed to evaluate IVQD, if the distribution matters (as stated in the previous point), could it be more beneficial to modify these questions and make them less general rather than remove them?
5. Table 4 reports performance following UPD-specific training. The paper notes that such training may degrade performance on general tasks. However, the performance under the *Orig* condition remains the same as indicated in Table 1. To provide a complete analysis, it would be helpful to re-evaluate the *Orig* condition after fine-tuning to observe any potential impacts.

**Minor Points:**

1. Fix the typo: *withcurrent* (line 135).
2. Consider reformatting Figure 2 and increasing the text font size for improved readability.
Yes
No ethics review needed.
1. **Manual Dataset Curation Standards**: The benchmark dataset involves certain manual efforts, such as eliminating image-agnostic questions and removing ambiguous questions. Could the authors clarify the criteria used for these removals? What standards were applied to ensure consistency and objectivity in these decisions?

2. **Alternative Solutions for UPD**: Are there potentially more advanced or effective solutions for addressing the challenges of Unsolvable Problem Detection (UPD) beyond those proposed? If so, what approaches could further improve MLLMs' performance in handling UPD scenarios?

3. The error analyses in section 6.2 is insightful, could you do the same analyses for open-source MLLMs?
1. The authors investigate the important issue of Unsolvable Problem Detection (UPD) and provide an in-depth analysis of this challenge within the context of multimodal large language models (MLLMs). 

2. To evaluate MLLMs' effectiveness in handling UPD, they develop a benchmark that incorporates diverse evaluation settings, examining various scenarios and testing a range of both open-source and closed-source MLLMs. The findings, particularly those outlined in Section 5.2, offer valuable insights into current MLLMs limitations.

3. To address the UPD challenge, the authors propose several practical strategies, including chain-of-thought prompting (CoT), self-reflection, and instruction tuning, which show promise in mitigating the difficulties MLLMs face in detecting unsolvable problems.
This paper addresses a challenge in multimodal large language models (MLLMs): Unsolvable Problem Detection (UPD). The authors conceptualize UPD through three distinct tasks: Absent Answer Detection (AAD), Incompatible Answer Set Detection (IASD), and Incompatible Visual Question Detection (IVQD). To systematically evaluate MLLMs’ capabilities in addressing UPD, they introduce a benchmark, MM-UPD (derived from MMBench), and assess both open-source and closed-source models on this benchmark. The evaluation reveals that current models exhibit substantial difficulty in handling UPD tasks. To mitigate these limitations, the authors propose a series of enhancement strategies, including chain-of-thought prompting, self-reflection, and instruction tuning, aimed at improving MLLMs' effectiveness in detecting unsolvable problems.
1. **Narrow Definition of Unresolvable Problem Detection**: The proposed definition of Unresolvable Problem Detection (UPD) is limited, focusing solely on multiple-choice problems. This approach overlooks other open-ended multimodal question-answering scenarios, which are prevalent and significant in real-world applications. Expanding the scope to include these scenarios would provide a more comprehensive understanding of UPD.

2. **Anticipated Poor Performance of Existing MLLMs**: The observed poor performance of existing multimodal large language models (MLLMs) in handling UPD is not unexpected. This is likely due to the absence of specific instruction-tuning datasets during the visual instruction fine-tuning stage. While the proposed solutions, such as chain-of-thought prompting, self-reflection, and instruction tuning, are steps in the right direction, they appear somewhat straightforward. More sophisticated and innovative approaches are needed to effectively address the complexities of UPD.

3. **Limited Benchmark Diversity**: The benchmark used for evaluating MLLMs' capability in handling UPD lacks diversity, relying solely on MMBench, which comprises 2,095 questions. This limited dataset may not provide a comprehensive evaluation. The exclusion of other datasets like MMMU and MathVista, based on their low standard accuracy, may not be a sufficiently robust justification. Incorporating a variety of datasets would enhance the evaluation's robustness and offer a more thorough assessment of MLLMs' performance in UPD tasks.
Yes
No ethics review needed.
What is the difference between the exsiting definition of unsolvable problems and the definition in this paper?
What is the difference in the design of Table 3 and Table 4, and why not merge them？
It first introduces a new challenge for LMMs, termed Unsolvable Problem Detection (UPD).
It constructs a benchmark to assess the performance across various ability dimensions.
It evaluates state-of-the-art LMMs on the UPD problem with the benchmark.
This paper introduces a new challenge for LMMs, termed Unsolvable Problem Detection (UPD) which examines the LMM’s ability to withhold answers when faced with unsolvable problems. Then, a benchmark is introduced to assess the performance across various ability dimensions and some solutions are explored to understand UPD. It can enhance the development of more practical and reliable LMMs.
The dataset in the paper is constructed based on existing datasets, and there is no comparative analysis of data size, distribution, and comprehensiveness, etc.
Large Multimodal Models; Benchmark ; Trustworthy AI
@inproceedings{
anonymous2024unsolvable,
title={Unsolvable Problem Detection: Evaluating Trustworthiness of Large Multimodal Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=K4YMFdx2Z2},
note={under review}
}
This paper introduces a novel and well-defined challenge for Large Multimodal Models (LMMs), termed Unsolvable Problem Detection (UPD). UPD examines the LMM's ability to withhold answers when faced with unsolvable problems. UPD encompasses three problems: Absent Answer Detection (AAD), Incompatible Answer Set Detection (IASD), and Incompatible Visual Question Detection (IVQD), covering unsolvable cases like answer-lacking or incompatible choices and image-question mismatches. In this paper, we introduce the MM-UPD Bench, a benchmark for assessing performance across various ability dimensions. Our experiments reveal that even most LMMs, which demonstrate adequate performance on existing benchmarks, struggle significantly with MM-UPD, underscoring a novel aspect of trustworthiness that current benchmarks have overlooked. To deepen the understanding of the UPD, we explore various solutions, including chain of thought, self-reflection, and instruction tuning, and demonstrate each approach's efficacy and limitations. We hope our insights, together with future efforts within the proposed UPD settings, will enhance the broader understanding and development of more practical and reliable LMMs.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Unsolvable Problem Detection: Evaluating Trustworthiness of Large Multimodal Models
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/f0b624d5710a329e6b5c13d801035afc78834d45.pdf
datasets and benchmarks
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/a076466cb80093888034589e3c9702ec544152d9.zip
Unsolvable Problem Detection: Evaluating Trustworthiness of Large Multimodal Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission