F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI
Explainable AI;Evaluation Metric;Computer Vision
Yes
No ethics review needed.
Questions are as below

#### W1:
- a) Provide specific examples of how "removal" is implemented for different data types like images, tabular data, and text.
- b) Give a more detailed explanation of how bounding the removed portion to a fraction of the input, rather than the explanation, helps mitigate OOD issues.

#### W2:
- a) Elaborate on how the random masking process relates to or simulates small perturbations, particularly in the context of image data.
- b) Provide examples or visualizations that demonstrate how this masking approach compares to traditional perturbation methods.
- c) Present empirical evidence or theoretical justification for why their masking approach is an effective way to improve classifier robustness.

#### W3:
- a) Provide empirical evidence or theoretical justification for why their fine-tuning process preserves the original classifier's decision boundaries.
- b) Compare the original and fine-tuned classifiers' outputs on a test set to demonstrate the degree of change.

#### W4:
- a) Provide theoretical or empirical evidence demonstrating the equivalence between explanations of the original and fine-tuned classifiers.
- b) Include a comparison of explanations generated for both classifiers on a set of test examples.

#### W5:
- a) Provide a step-by-step description of how they generate or obtain the ground truth (GT) explanations, including any assumptions made in this process.
- b) Include examples of GT explanations for different data types.

#### W6:
- a) Provide a detailed analysis of why F-Fidelity underperforms on this specific dataset and model combination.
- b) Investigate potential reasons, such as dataset characteristics, model architecture specifics, or hyperparameter settings that might contribute to this result.
- c) Discuss the implications of this finding for the broader applicability of their method.
- d) Consider conducting an ablation study to isolate the factors contributing to this performance difference.

The paper addresses an important aspect of fidelity evaluation and the issue of OOD while doing it. Overall, I like the central theme of the paper and that’s why I am leaning towards mild acceptance but I would like the authors to address the points in the weakness section and the questions in this section.
1) The paper deals with important aspect of fidelity evaluation
2) The authors discuss about OOD issue during fidelity evaluation
3) The authors propose an approach to handle the issues and show the performance in different data types like images, time series and NLP.
The authors propose a fidelity evaluation framework which utilizes an explanation agnostic fine tuning and a random mask generator which ensures that the generates input is not OOD. The efficacy of the proposed method if shown in various data types like images, time series etc.
W1) In section 3, it is mentioned in lines [215 to 222] that the size of the removed part is upper bounded by a fraction of the input and not a fraction of the explanation. It would be helpful to the readers if the authors could explain what they mean by "removed," as it might mean different in tabular data and images. Further, the authors should also explain how their strategy of selecting a fraction of input and not explanation addresses the OOD issue. 

W2) In lines[224:235], the authors use fine tuning to address the problem of robustness of the classifier for small perturbations(mentioned in lines [213:214]). However, at least from the perspective of images, it is unclear how applying a mask of {0,1} on random patches of an image can be considered as a perturbation.

W3) Further, the fine-tuning process mentioned in lines[224:235] seems like retraining the classifier with occlusion (for image classifiers). The authors should explain how their method won't change the decision boundary drastically so that their approach can be called fine-tuning and not training a different classifier altogether.

W4) The authors should explain why explaining the fine-tuned classifier is equivalent to explaining the original classifier (as mentioned in line[224])

W5) For the ease of readers, the authors should explain more about GT and how they extract it(in Section 4)

W6) It needs to be clarified to see why F-Fidelity performs worse than vanilla Fidelity and R-Fidelity for SST2 dataset with LSTM (Table 8). The authors are requested to explain this phenomenon in detail for the ease of readers.
Yes
The ablation study provided regarding B selection suggests this value can approach 1 (i.e. this is the same as if the value was not implemented) and the correlation scores would be good. Is this true? If so, what use does B have? In addition, what value of B is used and how was it selected? 

What is the intuition for fine tuning with the image perturbation limitation? I do not see why fine-tuning under image perturbed by random masks instead of perturbed explanations is better either. It is not obvious that this skirts the OOD problem. 

What is the value of being able to extract the explanation size? The motivation for this is lacking and it is not clear why it is important or interesting.
No ethics review needed.
The ablation study provided regarding B selection suggests this value can approach 1 (i.e. this is the same as if the value was not implemented) and the correlation scores would be good. Is this true? If so, what use does B have? In addition, what value of B is used and how was it selected? 

What is the intuition for fine tuning with the image perturbation limitation? I do not see why fine-tuning under image perturbed by random masks instead of perturbed explanations is better either. It is not obvious that this skirts the OOD problem. 

What is the value of being able to extract the explanation size? The motivation for this is lacking and it is not clear why it is important or interesting.
From their experiments which cover a broad range of applications and datasets, it seems this new method does improve over existing methods in rank correlation tests, indicating its ability to fairly and consistently sort attributions under evaluation. 

The solutions presented are simple.
The authors propose a new attribution evaluation framework which attempts to solve existing issues in current metrics such as the OOD problem from perturbation metrics, and the information leakage of fine-tuning methods like ROAR. To do this, the authors propose limiting the size of the perturbations made to each image and performing fine-tuning on images randomly masked under this perturbation size limitation.
I do not think the intuition behind the solutions is entirely clear and I do not think the presentation of the solutions is clear.  

The choices for multiple hypermeters are not indicated. 

More attribution methods should have been employed for a better representation of the existing space.
Yes
No ethics review needed.
1. I couldn't find how Pan et al., 2021, or Zhu et al., 2024 utilized macro correlation in their work. Can you explain how they employed macro correlation and how it relates to your citation of their work?

2. In Section 6, it seems that both MoRF and LeRF conditions would be satisfied equally. This is because deleting or inserting features based on the gamma value would maximize the MoRF-LeRF difference. I'm uncertain whether this experiment effectively demonstrates that F-Fidelity can distinguish explainers that are closer to the ideal Shapley explainer than other metrics. Could you clarify how this experiment supports that claim?

3. Regarding computational resources, I would like to know how much computational effort your method requires. One of the primary reasons ROAR isn't widely adopted is not just due to information leakage but also because of its significant computational cost. It would be helpful if you could provide details on the computational resources required for F-Fidelity and how they compare to those needed for ROAR.

** Typo : in figure 1, the y-axis label is RFid.
The paper is well written, especially introduction section and preliminary section.

The paper effectively tackles the OOD problem inherent in traditional removal-based evaluation methods. This is a crucial issue because OOD inputs can lead to unreliable assessments of an explainer's faithfulness.

The authors conducted extensive experiments across broad data, such as images, time series, and natural language processing.

The method is well mitigate limitation of prior methods; Addresses the information leakage issue and OOD issue at once.

The paper provides theoretical analysis demonstrating that F-Fidelity can recover the sparsity of the ideal SHAP explanations.

The paper introduces a novel approach by assuming that an explainer is better the closer it is to an ideal Shapley value explainer. This assumption is appealing because Shapley values provide a theoretically sound basis for feature attribution.
The authors presented F-fidelity, a new metric for evaluating the faithfulness of attribution methods. Traditional evaluation methods such as MoRF and LeRF suffer out-of-distribution(OOD) issue, and ROAR method suffers information leakage issue. F-fidelity overcomes these limitations through two strategies. 1. explanation-agnostic fine-tuning, which fine-tunes the model using stochastic masking operations. 2. a controlled random masking operation to overcome the OOD issue, by applying a random masking operation conditioned on the explainer's output so that removal of input features does not produce OOD samples. The authors conducted controlled experiments with other explainers. Moreopver, they showed that F-fidelity can be used to determine the sparsity of influential input components.
Lack of Visualizations Demonstrating F-Fidelity's Effectiveness. The paper does not include figures or visualizations that illustrate the effectiveness of F-Fidelity. Providing attribution maps or examples where there is a significant discrepancy between F-Fidelity and other evaluation metrics would enhance understanding and make the results more tangible. Visual comparisons could help readers better appreciate the advantages of F-Fidelity over existing methods. For example, Attribution map between samples that have discrepancy between F-Fidelity and MoRF - LeRF.

Novelty issue. Though theoretical availability of extracting the size of the ground-truth explanation is novel, the core difference between F-fidelity and R-Fidelity is introducing an upper bound on the fraction of input elements removed. This extension seems an incremental refinement rather than a fundamental novel idea.

Limited Explanation of Ground Truth. The paper's approach to establishing ground truth rankings involves using Integrated Gradients with systematically added noise levels. However, Integrated Gradients is known to be less faithful than some state-of-the-art explanation methods, such as the LRP (Layer-wise Relevance Propagation) family or LayerCAM. This raises concerns about whether the ground truth used is an appropriate benchmark. As the authors discuss ideal Shapley value explainers in Section 5, utilizing Shapley values as the ground truth might have been more appropriate to align with their theoretical framework.

Insufficient Clarification on the Importance of Macro Correlation. The paper could benefit from a more thorough explanation of why macro correlation between different evaluation methods is important. It was not immediately clear how macro correlation demonstrates the usefulness of F-Fidelity. Including figures or visual aids that depict the relationship between macro correlation and the effectiveness of F-Fidelity would help clarify this point and strengthen the argument.

Limited Variety of Explanation Methods Evaluated. The experiments involve a relatively small number of explanation methods. Comparing F-Fidelity scores across a wider range of explainers would provide a more comprehensive evaluation. For instance, methods like LayerCAM and LRP family are known to have higher fidelity than GradCAM, and GradCAM typically outperforms the pure gradient. Demonstrating whether F-Fidelity can reflect these known differences in fidelity would enhance the paper's validity.

Practical Challenges in Determining Explanation Size (c₁). In the section 6, the size of the most influential tier (c₁) is assumed to be known. However, in real-world applications, c₁ is not readily observable. It would have been beneficial if the authors had demonstrated how c₁ could be estimated or inferred from practical data, such as through experiments on datasets like ImageNet. Providing practical guidance on determining c₁ would make the theoretical contributions more applicable.
@inproceedings{
anonymous2024ffidelity,
title={F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable {AI}},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=X0r4BN50Dv},
note={under review}
}
With the rapid development of eXplainable AI, various methods are proposed to explain attributions, such as Integral Gradient, and SmoothGrad. However, how to measure the faithfulness of explanations maintains an open question. The most popular removal strategy suffers from the Out-of-Distribution(OOD) problem. The RemOve And Retrain and Importance Measure try to solve the OOD problem by retraining while suffering information leakage and convergence problems. To address these problems and provide a robust evaluation method for faithfulness measurement, we propose a new method, Fine-tuned Fidelity(F-Fidelity).  It alleviates the OOD problem by using consistent augmentation operations in fine-tuning and evaluation stages to reduce the gap between the training set and evaluation inputs. To verify the effectiveness of F-Fidelity, we proposed a fair comparison strategy employing various degraded explanations. We have conducted experiments on Image and Natural Language Processing classification tasks with two datasets and two architectures for each task. The results demonstrate the generality, and robustness of F-Fidelity.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Explainable AI Evaluation Metric Computer Vision
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/38590a17abfddbb809d48825b21ea22b70b42b4b.pdf
interpretability and explainable AI
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/92a6ad21a25177ce0045b44f48d78eaf811fabc1.zip
F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission