Halton Scheduler for Masked Generative Image Transformer
Image Synthesis;MaskGIT;Halton Sequence
Yes
No ethics review needed.
NA
This paper addresses an overlooked problem: choosing which tokens to decode during masked image modeling. The intermediate analysis and motivation are clear, and the proposed solution is promising while being simple. Comparison against random decoding is also conducted.
This paper proposes a new schedule for token decoding in masked image modeling. MaskGIT decodes high confidence tokens first. Instead, this paper shows that this technique is not optimal and can lead to decreased performance with more decoding steps which should not occur with a good schedule. To combat this, the paper proposes a Halton schedule based on mutual information analysis to spread out the predicted tokens. The technique requires no training and can be plugged into other models. Experiments are conducted on class conditional and text conditional image generation.
I generally like this approach. The problem and motivation are clear, and the solution is simple and seems promising. However, some of the results are not convincing enough, and the presentation of the paper is unpolished and could benefit from a few rounds of editing.

- It is unclear to me how to interpret Table 1, because the baseline is from the paper for which no code exists. The models are very different parameter sizes, comparison against random and confidence is missing. I would like to see more rows like in Table 2. In fact, Table 2 and 3 are more convincingly demonstrate the effectiveness of this approach. However, the image resolutions do not match and so it is really difficult to draw conclusions.
- In general, the experimental part of this paper is weak, because if true, much more models and tasks should have been evaluated to showcase the plug&play-ability of this approach for various MIM tasks and models. From the current results, it is not clear how well the approach generalizes across datasets, models, and tasks.
- Does the method have any limitations, weaknesses or failure cases? What about runtime and memory complexity compared to random and confidence scheduling?
- Fig 5 should not be a line plot in my opinion as the x-axis is categorical.

In summary, the method is promising, and the paper presents an initial good signal. I encourage the authors to work on a more comprehensive evaluation of MIM models, tasks, and datasets to demonstrate the benefits of this model and produce fair and clear comparisons on fixed settings. As of now, unfortunately, there is not enough evidence to support the claims made convincingly.
Yes
No ethics review needed.
I don't have a good sense of what is the extra information that is being learnt using Halton schduler and the intuition is not very clear. I agree with the analysis of the paper but the proposed solution doesn't;t seem very well analysed and ablated.
1) Overall the idea of Halton scheduler makes a lot of sense and addresses the important problem of scheduler in MaskGIT kind of models.
2) Halton scheduler proposes a improved schedule which makes the model improved generation capabilities.
3) Experimental results show pretty convincing improvements across text-to-image zero-shot COCO, MUSE , MaskGIT and other competetive methods.
This paper presents the Halton Scheduler, which refines the token unmasking strategy in MaskGIT. The authors begin by analyzing the Confidence Scheduler previously designed for MaskGIT (Chang et al., 2022; 2023), which prioritizes the unmasking of the most certain tokens. They demonstrate that while this method enhances confidence, it can reduce diversity and image quality due to the clustering of tokens around already predicted regions. To address this, they propose a novel, deterministic scheduler that leverages the Halton low-discrepancy sequence (Halton, 1964). This approach is informed by understanding entropy progression during image generation, aiming for a more even distribution of token selection to ensure uniform coverage across the image. They show convincing results over baselines like MUSE and MaskGIT across image generation and text to image generation.
Weakness:
1) How was the discretization of the sequence Hnh into a grid corresponding to the token map done? This seems to be an approximation which I’m not sure about.
2) More intuition and details need to be added especially for Halton scheduler, like the Low-Discrepancy properties need to be discussed more. 
3) Discussion on improved halton sequencing needs to be added. Using the Scrambled Halton Sequence, the Leapfrog Method, and Higher-Dimensional Generalizations need to be discussed. 
4) Similar ideas for information overload into tokens have been studied in the registers[1]. Analysis on token norm in this setting will be useful and could reflect whether there is similarity in understanding between high-norm tokens and the proposed scheduler. 
5) Is Extra compute time required to use Halton scheduler? One of the main advantages of the Halton scheduler is the speed of image generation. Would like to see what is the inference time with Halton's scheduler?
References
1) Vision Transformer needs registers.
Yes
No ethics review needed.
- After vector quantization, do token relationships still correspond to their Euclidean distance?
- Might this uniformity lead to the overlooking of critical details in specific areas, thus negatively impacting sampling accuracy?
- Presents an innovative mutual information analysis that clarifies the scheduler’s role in MaskGIT.
- By leveraging the Halton sequence, the method ensures a more even distribution of selected tokens, grounded in the information gained at each step and the mutual information between tokens.
- Enhances both image quality and diversity in class-to-image and text-to-image generation.
This paper proposes a novel scheduler based on the Halton sequence, aimed at minimizing the correlation between simultaneously sampled tokens and maximizing the information gained at each step. By selecting token positions using a quasi-random, low-discrepancy Halton sequence, the approach reduces FID scores and improves both image quality and diversity.
- The assumption for Eq. 5.b is not sufficient, which ignores the long-distance dependencies.
- The influence of conditional entropy is not limited to spatial proximity; semantic information and context play a significant role and should be considered.
Yes
No ethics review needed.
NA
NA
This paper proposed a new sampling strategy using the Halton scheduler which uniformly recovers the image spatially. Several experiments are conducted to validate the effectiveness.

**Additional Comments**:

The manuscript does not follow the margin of ICLR template, which violates the submission guidelines. 

In all, I lean towards desk rej.
- The paper is poorly written. There are tons of typos and layout issues making it hard to follow.
- The contribution is not enough. It is like a simple spatial prior to the sampling process.
- The experiments are limited and unfair. For example, in table 1, the author compared the proposed method with the baseline MaskGIT. However, the "Ours" baseline is not presented. The compared MaskGIT has a different # param compared to the proposed approach.
- Fig 5 is very confusing. It is not understandable. Why is only the FID/IS reported for AR mode? 
- In table 2, the results are not fair. As compared MaskGIT* is not trained to predict images in 32 steps.
- The qualitative results are also limited. What are the text prompts for fig 8?
- Fig 7 is very confusing. Is the training target of the confidence scheduler set to be 32 during training?
We propose a new scheduler for MaskGIT leveraging the Halton sequence, ensuring uniform token coverage and enhancing quality and diversity without noise injection.
@inproceedings{
anonymous2024halton,
title={Halton Scheduler for Masked  Generative Image Transformer},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=RDVrlWAb7K},
note={under review}
}
Masked Generative Image Transformers (MaskGIT) have emerged as a scalable and efficient image generation framework, able to deliver high-quality visuals with low inference costs. However, MaskGIT's token unmasking scheduler, an essential component of the framework, has not received the attention it deserves. We analyze the sampling objective in MaskGIT, based on the mutual information between tokens, and elucidate its shortcomings. We then propose a new sampling strategy based on our Halton scheduler instead of the original Confidence scheduler. More precisely, our method selects the token's position according to a quasi-random, low-discrepancy Halton sequence. Intuitively, that method spreads the tokens spatially, progressively covering the image uniformly at each step. Our analysis shows that it allows reducing non-recoverable sampling errors, leading to simpler hyper-parameter tuning and better quality images. Our scheduler does not require retraining or noise injection and may serve as a simple drop-in replacement for the original sampling strategy. Evaluation of both class-to-image synthesis on ImageNet and text-to-image generation on the COCO dataset demonstrates that the Halton scheduler outperforms the Confidence scheduler quantitatively by reducing the FID and qualitatively by generating more diverse and more detailed images. Our code is publicly available on [link redacted].
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Image Synthesis MaskGIT Halton Sequence
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/776ff484195cdf34e33a3a22fc903ee32cbddcc1.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/f3cba3e79bade301fcb6ac731f5275432807baae.pdf
Halton Scheduler for Masked  Generative Image Transformer
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission