Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Modulating Individual Human Percepts
Perceptual variability;Diffusion model;Object Recognition;Behavior Modulation;Behavioral Alignment
Yes
No ethics review needed.
I have several questions which I would like the authors to respond to. I have stated them in the 'Weaknesses' section of the review, but I will list them here for the authors' convenience:
* How many of the 4741 were generated using ’0’ or ‘1’ as the reference image? How many of these images were generated by uncertainty guidance and how many by controversial guidance?
* How statistically significant are the claims of randomness of the distribution of human prediction accuracy?
* How are images to be displayed to subjects selected? What design choices were made to ensure that subjects were equally likely to view images generated by different reference images?
* What is the correct label for an image from the varMNIST dataset? What was the rationale for the choice you made?
* In finetuning the models, what are the targets used in the case of population-level alignment?

In addition, I have a couple of additional questions for which I would appreciate the authors' clarifications:
* Why is the MSE loss important? The authors claim that bias in the prior distribution of MNIST causes generated images to be biased towards specific digits. Would this not be solved by sampling from MNIST so that all digits are equally likely to be selected as reference images?
* Why is model entropy (Fig 4a) centered around 0?
* Will the varMNIST dataset along with the results of the behavioral experiments be made publicly available?
The paper is well-motivated by existing work in cognitive science looking to study human perception with ambiguous stimuli. It also follows a recent body of work using AI models to study human perception. In this regard, the paper is tackling an interesting and important problem. There are several strengths of this paper, the most important of which are described below:

* The synthetic visual stimuli generated by the diffusion model using uncertainty and controversial guidance are of high quality — examples in the paper are stylistically similar to real digit-like images while evoking confusion among human subjects. Further, there seems to be significant variation in the degree of confusion (measured by entropy of human predictions) generated by the images. This kind of variability is highly desirable for such datasets which are intended for cognitive science research into human perception. 
* Analysis in the paper shows that there is some degree of correlation between the entropy of visual AI classifiers and human classifiers on the images from the varMNIST dataset. This shows that the (uncertainty or controversial)-guided diffusion process generates synthetic images which capture some aspects of human perceptual boundaries.
* The authors’ use of the varMNIST dataset and the predictions from subjects to generate controversial stimuli that uncover individual differences amongst subjects is fascinating. 
* The authors do a good job situating the paper in the context of existing work. The Introduction and Related Work sections do a good job putting this work in the context of existing research, while the Discussion section does a good job describing the contributions, implications and limitations of the work.
The aim of this paper is to generate ambiguous digit-like images to study perceptual processing in humans. To this end, the authors propose a diffusion-based method for generating counterfactual digit-like images that produce conflicting predictions from humans; the diffusion process is guided by two kinds of signals, termed the uncertainty signal and the controversial signal. The uncertainty guidance encourages the diffusion model to produce images for which an AI visual classifier is confused between two labels for the images, while the controversial guidance encourages generation of images where two different classifiers produce diverging labels for each image. Further, the diffusion is guided by a mean-squared error loss to encourage generated images to be like real handwritten digits. This method is shown to be effective, as the generated images are stylistically similar to real images, but are confusing for humans to classify. Behavioral experiments conducted by the authors corroborate these claims. The authors term these generated images, together with the human classification data, the varMNIST dataset. Analysis of human prediction patterns for images from the varMNIST dataset shows slight positive correlation between an AI classifier’s prediction entropy and the entropy of human predictions on varMNIST images. By training AI classifiers using the human predictions, the authors showed that the varMNIST dataset can be used to identify controversial stimuli which reveal patterns of individual differences between subjects in predicting such ambiguous images.
* The paper only considers the generation of ambiguous digit-like images, whereas for the purpose of human perception, natural images are much more relevant. In fairness, this weakness is acknowledged by the authors; nevertheless it remains a weakness.
* Important implementation details, such as the network and architecture details for the image generation, are missing in the paper. For the diffusion process itself, the appendix states that the authors used a pre-trained diffusion model. However, the architecture and the pretraining dataset are not described. Further, for the guidance signals, the architectures of the classifiers $f_1$ and $f_2$ have not been discussed. This significantly hampers both the reproducibility of the paper and the soundness of the work. 
* Behavioral experiments require carefully designed checks to ensure that the dataset is balanced and not biased towards specific stimuli. Further, statistical analysis must be done after data collection to ensure that no bias inadvertently makes it into the dataset. The paper is lacking on both those counts. Some of these concerns are listed below:  
    * The distribution over the 10 digits to be reference images in the final 4741 images is not presented. How many of the 4741 were generated using ’0’ or ‘1’ as the reference image? How many of these images were generated by uncertainty guidance and how many by controversial guidance?
    * The authors claim that the generated images have a random distribution over the range of entropies. This claim however is not supported by statistical tests. The histogram can be misleading since they are notoriously sensitive to the choice of parameters like the bin size.
    * In the random trials, how are the images to be displayed to subjects selected? Were there choices made to ensure that a subject sees the same number of synthetic images generated by using ‘0’ as the reference digit as the number of synthetic images generated by using ‘1’? Any kind of accidental subject-level bias can strongly affect patterns of individual differences studied later in the paper.

* The description for the finetuning process was quite unclear. 
    * Were the labels of the reference images used as the correct labels to calculate the accuracy on varMNIST? I’m not sure about such a choice because the motivation for the dataset is to generate ambiguous images and having fixed labels for such images seems counterintuitive.
    * What are the targets used to finetune the models when using population-level data? Is it the average over predictions for a particular image over all subjects?
 
* The writing in the paper can be improved to improve clarity and readability of the paper.
    * One of the main contributions of the paper is the varMNIST dataset and the diffusion process used to generate the images. It is surprising that a lot of highly relevant details regarding the image generation are included only in the appendix. I would strongly recommend the authors to update the manuscript to include pertinent details, at least briefly, in the main paper.
    * The same point holds about the behavioral experiments. Details such as the number of subjects, number of trials and rationale for choosing the criteria for selecting the 4741 images are better placed in the main paper because they are central to the paper’s contributions.

* A couple of minor typos:
    * perveptual -> perceptual in section 5 title (line 310)
    * There is no section 3.1 (line 402)
Yes
No ethics review needed.
See Weaknesses
This paper presents an innovative approach to studying human perceptual variability by generating synthetic MNIST images from ANN perceptual boundaries, which are useful for extensive human behavioral experiments.
This paper presents a novel approach to studying human perceptual variability using artificial neural networks (ANNs). The authors develop a generative model that samples from the perceptual boundaries of ANNs to create synthetic MNIST images, inducing high variability in human perception. This method facilitates the construction of the varMNIST dataset, which is used for behavioral experiments involving 346 participants and over 240,000 trials. The research highlights that aligning the perceptual variability between ANNs and humans can predict human decision-making.
1. Scope of the dataset: The proposed method is only evaluated on MNIST, which limits the generalizability of the findings to real-world applications. While MNIST is effective for the initial exploration of perceptual variability, it does not capture the complexity inherent in more diverse visual datasets. This restricts the study's ability to address more nuanced or complex perceptual phenomena that may arise in real-world visual recognition tasks. For instance, datasets that include natural scenes, complex objects, or varied lighting and perspectives could provide more robust insights into human perceptual variability and ANN alignment.
2. Lack of quantitative comparison to baseline methods: Although the paper discusses the limitations of baseline methods in the related work section, it does not provide a quantitative comparison to those baseline approaches. This omission makes it difficult to objectively assess how much better sampling from the perceptual boundaries of ANNs performs in terms of aligning with human perceptual variability. A quantitative comparison would strengthen the paper's claims by demonstrating measurable improvements over baseline methods.
3. Trade-off between alignment and performance: While the paper shows that fine-tuning ANNs with human behavioral data improves perceptual alignment, there is a trade-off in performance on standard classification tasks, as pointed out by the authors. This suggests that optimizing models for perceptual alignment with human behavior may come at the cost of reducing their effectiveness for standard tasks.
Yes
No ethics review needed.
- About the analysis of the performance of the finetuned models (Fig. 5)
    - Was the varMNIST dataset split into a training and test set? Or were the models evaluated on the same images and labels on which they were finetuned? In other words, how can we be sure that the models are not overfitting on the individual subject data?
    - Does the "subject-finetuned (varMNIST-subject)" evaluation mean that each model that was finetuned on an individual subject's data was then tested on the data of that subject?
- Figure 6b: for some of the subjects, the subject-finetuned model performs worse at predicting in-subject compared to in-cluster. What is the explanation for this?
- How was the parameter $\alpha$, which trades off the entropy or controversy loss with the MSE loss, set?
- "we found that experimental images with response times between 300 and 5000 ms and entropy values between 0.5 and 2.5 best met our experimental requirements" (l. 716-717). What exactly were the experimental requirements?
The idea of generating a dataset with the goal of having high variability between human behavioral responses is compelling and the paper nicely combines controversial stimuli and diffusion-based image generation to achieve this. I am very fond of the general approach of analyzing inter-individual variability instead of simply using neural networks to predict average human behavior and I think this is a promising direction. Overall, I found the paper well written and easy to follow. It starts with a clear introduction and description of what it sets out to do and Figure 2 really helps to understand the general idea of the dataset generation.
The paper is concerned with understanding variability in perceptual judgments between different people. It introduces a method for generating controversial stimuli, i.e. stimuli that elicit different classifications from different neural networks, in order to generate a dataset of human behavioral responses with high variability (varMNIST). Different classifiers are then finetuned on individual subjects' responses in varMNIST. These subject-specific classifiers are then used to generate a new set of controversial stimuli tailored to differences between individual subjects.
Unfortunately, I found it hard to understand some of the specifics of the methods and analyses, leading to several points of confusion and remaining questions, which are listed below and under "Questions". This also affects the reproducibility of the results, which I think would be really difficult to do just from the description of the methods in the paper, especially without any code.

Furthermore, I am not convinced that the results presented in the paper actually deliver on some of its central claims, that the generated images "more closely resemble natural images" or that the authors successfully "modulated individual behavior, unveiling significant inter-individual differences in perceptual variability". Details on these points also follow below. I admit that some of this might be due to me misunderstanding details of the analyses performed in the paper, which the authors might be able to clarify.

1. At first, I was confused about the dataset construction, in particular about what is taken as "ground truth" for varMNIST. After reading Section 3 "Human experiment and dataset construction", I got the impression that varMNIST contains 4,741 images, each with a label. After looking at Fig. 5b and reading Appendix A.3 and A.4, I think I understood that varMNIST contains a data point per subject per image, resulting in  102,021 labels in total. This should be clarified in the main text.
2. The abstract states that varMNIST contains 242,900 images, while Appendix A.4 suggests that it contains 102,021 after additional filtering. It is not clear which number applies to the actual dataset.
3. The discussion claims that "by utilizing carefully designed controversial stimuli, we selectively modulated individual behavior". I assume that this is about the controversial stimuli that were designed based on the subject-finetuned models in Section 6. I agree that this kind of analysis is a promising idea to investigate inter-individual variability in behavior. However, if I understand correctly, the newly generated stimuli are not actually tested in behavior. The way it is currently written, the paragraph "Controversial stimuli between subjects exhibit typical patterns" is misleading. It reads as if it is about disagreements between subjects in the experiment, but it is actually based on disagreements between models (which is clarified in the caption for Fig. 7). This is problematic because the subject-finetuned models seem to have a very low accuracy for quite a few subjects (see Fig. 5b). This should be clarified in the main text and the limitations of the analysis should be discussed. Whether the controversial designed for the subject-finetuned networks actually agree with the perceptual judgments of those subjects is an empirical questions that could be tested with a follow-up experiment. 
4. The discussion of related work in the Methods section "Diffusion model as a regularizer to introduce prior information" seems a bit imprecise. In particular, it states that "a common issue was that the generated images lacked sufficient naturalness and failed to significantly influence human perception". But taking a quick look at one of the cited references (Gaziv et al., 2024), it seems that the generated images, while not necessarily being natural, did significantly influence human perception. The same is true for some of the other cited studies.
5. The claim that the proposed method results in images that "more closely resemble natural images" compared to previous methods for controversial stimuli is doubtful. Previous methods (e.g. Gaviz et al., 2024), which the authors of the present paper criticize as unnatural, are applied to photographic images like ImageNet. The present study is concerned with MNIST-style handwritten digits. It is not clear how the results transfer to photorealistic domains and if the diffusion-based approach would also lead to more "natural" images in these cases.
6. A couple of points about data visualization, which sometimes make it hard to follow the arguments made about the data.
    a. If I am not mistaken, Fig. 5 presents the same data in three different ways (average accuracy bar plot, single-subject accuracy point plot and kernel density). This could be simplified with something like a rainclould plot (https://wellcomeopenresearch.org/articles/4-63), which contains a point plot, kernel density estimate, and the average in a single plot.
    b. The color scheme is confusing. In Fig. 5a, different shades of red and blue indicate different models, while in Fig. 5b, some of the same colors are used to indicate training schemes. In Fig. 5c, the same traning schemes are then represented by different colors. In Fig. 6, again the same colors are used to represent out-cluster, in-cluster and subject-level predictions, and in Fig. 7 they represent different datasets. My recommendation is to use different colormaps to represent different concepts / distinctions.

Minor points:
- Fig. 7b is not referenced in the text
- The paragraph "Controversial stimuli between subjects constructed by finetuned models. " states that controversial stimuli were generated "using the algorithm in Section 3.1" (l. 401-402): There is no Section 3.1, so which algorithm was actually used?
Yes
Yes, Responsible research practice (e.g., human subjects, data release)
- The formulation in lines 185 and 192 seems inconsistent. The left-hand side introduces $f_{\phi}(\cdot)$, but the right-hand side does not include $f_{\phi}$​ or the parameter $\phi$. It would be helpful to rewrite these equations more formally.
- The cluster analysis results in eight distinct clusters, suggesting high perceptual variability among participants. However, what characteristics of the participants contribute to this variability? Is it due to demographic attributes, experimental environment, or purely biological differences? Additionally, why does the analysis yield eight clusters?
- When interpreting results from human experiments, statistical significance should be considered. For example, in Figure 5, the difference between the "finetuned" and "subject-finetuned" results is minimal, yet the paper describes this as a "significant difference in perceptual variability across participants." To ensure validity in human studies, a stricter approach with p-values would be appropriate.
Could you clarify when uncertainty guidance and controversial guidance are used? Are they applied simultaneously or separately to generate images? In line 255, the paper states that images are generated using both guidance methods. Does this mean they are integrated to create a single image, or are they used separately to generate different subsets of images? If the latter, has there been any quantitative ablation study comparing their effectiveness in generating ambiguous samples for human perception?
- As noted in the paper, many prior works on adversarial examples and counterfactual explanations also leverage diffusion models to create natural samples. This setup is not novel.
- The technical novelty of this work is limited. The use of classifier guidance in diffusion models for conditional generation is a well-known technique, and post-processing samples with human evaluation is a standard dataset curation method. While the paper highlights uncertainty guidance, controversial guidance, and the end-to-end system as contributions, the controversial guidance was previously introduced by Golan (2020), and uncertainty guidance resembles untargeted adversarial attacks, which also occur at the classifier’s decision boundary, yielding high-entropy predictions. From a systems perspective, the paper does not offer system-level engineering innovations to enhance the generation process. Therefore, the main contribution lies in the problem addressed and its application to cognitive studies of human perception.
- There is a typo in line 310: "perveptual" should be corrected to "perceptual."
1. The paper provides interesting results from human studies, particularly in aligning perceptual boundaries between ANNs and human perception.
2. It introduces a novel end-to-end system to generate datasets for cognitive studies on human perception and its relationship with vision ANNs.
3. The paper conducts diverse empirical evaluations, complemented by human studies, to analyze the proposed datasets and uncover insights into human perception.
This paper presents a counterfactual generation-based method to study subject-level perceptual variability in humans. To automatically and scalably synthesize stimulus samples for human studies, the authors propose a generative model that samples images from the perceptual boundaries of artificial neural networks (ANNs) and integrates a post-hoc data curation procedure involving human input. The generative model guides the diffusion model using two approaches: (i) uncertainty guidance, which maximizes the entropy of the ANN’s predictions, and (ii) controversial guidance, which maximizes the prediction discrepancy between two different ANN models. Using this method, the paper introduces a synthetic dataset, varMNIST, to systematically investigate subject-level perceptual variability in three areas: (i) human behavioral analysis in image recognition tasks that evoke perceptual variability, (ii) alignment between ANN and human perception, and (iii) subject-level perceptual variability in response to controversial stimuli.
1. The technical contribution of uncertainty and controversial guidance is marginal, as these concepts already exist in prior works, and the paper does not offer significant system engineering innovations.
2. The post-hoc analysis of human studies lacks rigor. The findings on human perception are not supported by statistical significance, and the empirical results are not discussed in depth. For example, the paper notes inter-subject and subject-group differences in controversial stimuli but does not explore the factors contributing to these perceptual differences.
By sampling along neural network perceptual boundaries, we generated natural images that induce high variability in human decision and can predict and modulate individual behavior on these samples.
@inproceedings{
anonymous2024synthesizing,
title={Synthesizing Images on Perceptual Boundaries of {ANN}s for Uncovering and Modulating Individual Human Percepts},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=APWIZgehDT},
note={under review}
}
Human decision-making in cognitive tasks and daily life often exhibits significant variability influenced by various factors such as task difficulty, individual preferences, and cultural differences. Understanding individual variability helps enhance our comprehension of the perceptual and decision-making mechanisms when humans confront uncertainty and ambiguous information. Here we present a counterfactual-based approach to investigate the subject-level decision-making behaviors and the underlying perceptual mechanisms using synthetic visual stimuli. We develop an efficient generative model that samples along the artificial neural network (ANN)’s perceptual boundary to create image samples capable of inducing high variability in human perception. These generated samples, combined with behavioral measurements collected from 346 participants with 242,900 trials, form the varMNIST dataset. By aligning the perceptual variability between humans and ANNs, we successfully predict human decision-making behaviors on varMNIST. Furthermore, we are able to selectively modulate individual behaviors by generating tailored controversial stimuli. These stimuli reveal large individual differences, indicating the inter-subject perceptual variability. Together, our study uncovers specific distinctions between humans and machines in the variability of their perceptual experiences and opens a new avenue for modulating individual decision-making behaviors, providing new perspectives for developing artificial intelligence models with personalized perception.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Perceptual variability Diffusion model Object Recognition Behavior Modulation Behavioral Alignment
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/a4182045f1c816a7c03e1bf4ec69ba7739083ba4.pdf
applications to neuroscience & cognitive science
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Modulating Individual Human Percepts
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission