Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scale
Mesh generation;3D Generation;Hourglass Transformer;Autoregressive mesh generation
Yes
No ethics review needed.
- How much does the number of points in the point cloud impact the final mesh generation process
- How does the model handle partial point clouds?  
- Above all, I would like the point regarding the comparision with [1, 2] to be addressed and would change my rating if it was a misunderstanding on my side.


[1] Shen, Tianchang, et al. "Flexible Isosurface Extraction for Gradient-Based Mesh Optimization." ACM Trans. Graph. 42.4 (2023): 37-1.
[2]Shen, Tianchang, et al. "Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis." Advances in Neural Information Processing Systems 34 (2021): 6087-6101.
The work is well ablated, and the choice in architectural design taken by the authors is reasonable and easy to understand. Overall, the work is well written and well structured, making it easy to understand why and what was chosen and how it was implemented. The appendix is additionally very useful to reimplement this work. Overall, the work seems to have well-reasoned analysis for training mesh generations from autoregressive models and additionally adds limitations to the mesh generation process that seemingly seem to be very impactful in this process.
The author presents a methodology for generating high-quality mesh generation from point clouds. The author proposes an autoregressive model that has been well evaluated and chosen regarding the explicit need for mesh generation. The authors claim well-analysed contributions in the chosen neural architecture and the training methodology and reconstruct the mesh by limiting the scope of possible solutions to enforce mesh sequences during training and inference.
I am not an expert in this research area, but for me, the main weakness is the claim that previous work "recent attempts at generating artist-like meshes are limited to 1.6K faces and heavy discretization of vertex coordinates". I want to refer the authors to works such as [1, 2] that show mesh reconstruction with more that 1.6K faces.
Both of the mentioned works [1, 2] show high-quality reconstructions and are not compared by the authors.

[1] Shen, Tianchang, et al. "Flexible Isosurface Extraction for Gradient-Based Mesh Optimization." ACM Trans. Graph. 42.4 (2023): 37-1.
[2]Shen, Tianchang, et al. "Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis." Advances in Neural Information Processing Systems 34 (2021): 6087-6101.
Yes
No ethics review needed.
I believe that the problem this paper aims to address (generating detailed meshes) is meaningful. However, there are many areas for improvement in the experimental design and writing.
The proposed method addresses an important challenge in 3D mesh generation: the number of faces significantly affects the quality and fidelity of the results, making a higher face count crucial. Experimental results indicate that this method can produce highly detailed meshes. 

Additionally, the authors present the motivation for their algorithm design, based on observed issues in real-world data, offering valuable insights for the academic community.
This paper presents a method for efficiently generating meshes from point clouds. The authors propose an hourglass transformer network architecture and a truncated sequence training scheme, which enable the generation of meshes with a large number of faces and diverse types. Compared to existing methods, this approach requires less training memory and achieves faster throughput.
The author mentions that the tokens of the last vertex have high perplexity, which led to the design of the Hourglass Transformer to address this issue. However, the causal relationship here does not seem entirely clear. Within the architecture of the Hourglass Transformer, there is no obvious special treatment for the last vertex tokens. While the Hourglass Transformer is an effective structure, linking it directly to the perplexity of the last vertex tokens feels somewhat tenuous.


In the experiments, the authors used high-quality point clouds as input. However, in practical applications, point clouds often contain noise. This paper does not discuss or experiment on this issue, which inevitably raises concerns about its practicality.  

The author only compares their method with one other approach and does not include a related work section. This makes it difficult for those not working in mesh generation to follow the advancements in the field and compare the proposed method. For instance, how does it differ from existing mesh generation methods such as CLAY[1]?

[1] Zhang L, Wang Z, Zhang Q, et al. CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets[J]. ACM Transactions on Graphics (TOG), 2024, 43(4): 1-20.
Yes
No ethics review needed.
Please see the weakness section.
The proposed approach shows potential. The intermediate latent representation makes use of the geometric construction of meshes. The training procedure allows the architecture to scale to longer and more complex meshes.
This paper introduces a transformer-based approach to mesh generation. Given an input point cloud, the hourglass architecture (MESHTRON), like an Unet or an AE, compresses and reconstructs the mesh as output. The intermediate states (or latent) have an explicit meaning, corresponding to vertex and face representation. During training the model is trained by exploiting the adjacency of the triangles combined with a sliding window approach. Such a choice reduces memory consumption and allows the model to scale to complex and longer mesh sequences.
Although the paper is very promising, several details are unclear. I would appreciate it if the authors clarify them:
1. several details are obscured in the manuscript, or not appropriately presented:
   - the meaning of the intermediate representation is not well presented. It can be inferred from Figure 4, but I the authors should to discuss it in the text for the sake of the reader - please expand Section 3.1;
   - it is not clear how the sliding window is used during inference. For example, how is the sliding window decided? And given that the sequence is truncated during training, how is a larger mesh generated? Please, provide more details on the sliding window inference process in Section 3.2.
2. the authors claim the length of the mesh sequence can be 64K in faces, L158-159. However, I cannot find such examples (even in the supplementary material). Is this a theoretical limit or do authors have examples and did not include them in the material?
3. the authors point out "MESHTRON does not work from time to time", would you please elaborate? This is not an appropriate sentence for a scientific paper.
4. how is the global conditioning obtained? Is the input point cloud feed to the model and then the sequence used for cross attention?
5. would it be possible to condition from an image latent representation (say encoded by DinoViT) and project it onto the latent space of MESHTRON? (it is fine if it is future work, but worth mentioning in the paper)
6. a comparison with MeshGPT would be appreciated even if qualitative. As the code is not available, the authors could maybe reach out to MeshGPT's authors to get some examples.
An autoregressive mesh generator based on Hourglass and sliding window attention that exploits the periodicity of a mesh sequence for better efficiency and exceptional performance.
@inproceedings{
anonymous2024meshtron,
title={Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scale},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=mhzDv7UAMu},
note={under review}
}
Meshes are a fundamental representation of 3D surfaces. However, creating high-quality meshes is a labor-intensive task that requires significant time and expertise in 3D modelling. While a delicate object often requires over $10^4$ faces to be accurately modeled, recent attempts at generating artist-like meshes are limited to $1.6$K faces and heavy discretization of vertex coordinates. Hence, scaling both the maximum face count and vertex coordinate resolution is crucial to producing high-quality meshes of realistic, complex 3D objects. We present Meshtron, a novel autoregressive mesh generation model able to generate meshes with up to 64K faces at 1024-level coordinate resolution --over an order of magnitude higher face count and $8{\times}$ higher coordinate resolution than current state-of-the-art methods. Meshtron's scalability is driven by four key components: 
(i) an hourglass neural architecture, 
(ii) truncated sequence training, 
(iii) sliding window inference, 
and (iv) a robust sampling strategy that enforces the order of mesh sequences.
This results in over $50\%$ less training memory, $2.5{\times}$ faster throughput, and better consistency than existing works. Meshtron generates meshes of detailed, complex 3D objects at unprecedented levels of resolution and fidelity, closely resembling those created by professional artists, and opening the door to more realistic generation of detailed 3D assets for animation, gaming, and virtual environments.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Mesh generation 3D Generation Hourglass Transformer Autoregressive mesh generation
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/12a33eaa589420871c1437f50b99fb6821c31831.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scale
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission