Looks Great, Functions Better: Physics Compliance Text-to-3D Shape Generation
3D shape generation;Functional 3D model;Physics perception;Differentiable physics layer;Solid mechanics
Yes
No ethics review needed.
I am interested in understanding how your Fun3D method compares to autoencoder-based 3D generation frameworks such as Craftsman and Clay, particularly in terms of physical characteristics. Do these frameworks already possess good physical properties, and if so, how does Fun3D's performance, especially in terms of physical functionality, qualitatively compare to these existing works? 

I am curious about the practical validation of the physical properties of the 3D shapes generated by the Fun3D method. Could you elaborate on how you demonstrate that these shapes not only adhere to theoretical physical principles but also exhibit good physical characteristics in practical applications? Specifically, I would like to know if you have conducted any simulations or real-world tests, such as finite element analysis (FEA) simulations or actual 3D printing, to validate the structural integrity and functionality of the generated shapes in real-world scenarios.
Fun3D's primary advantage lies in its integration of physics-based constraints, which ensure that the generated shapes are not only geometrically sound but also practically functional in real-world scenarios.


The method employs a data-driven differentiable physics layer that allows for the simultaneous optimization of shape geometry and its physical attributes. This layer is trained using physics priors embedded through Finite Element Method (FEM) data, which provides a high level of accuracy in simulating the mechanical properties of the 3D shapes.

Fun3D adopts an alternating training strategy that efficiently balances the optimization of geometry and physics, leading to shapes that are robust and better suited for applications such as engineering design and virtual prototyping.
The paper introduces Fun3D, a method for generating 3D shapes from text that ensures visual appeal and physical functionality. It features a data-driven differentiable physics layer for optimizing 3D shapes according to solid mechanics principles, providing the practicality of generated shapes for real-world applications.
Primarily, the physical considerations in Fun3D are somewhat narrow, focusing mainly on stress, strain, etc, which is not a general framework and may not fully capture the complexity of real-world physical interactions. This could potentially lead to 3D models that, despite conforming to the considered physical laws, still lack a broader range of physical realism. 

Additionally, the generated models' overall quality is reported to be average. The paper does not extensively discuss autoencoder-based 3D generation frameworks, which provide good geometric configurations and may already possess desirable physical properties. This omission raises questions about the comprehensiveness of the authors' exploration of the 3D generation landscape.
Yes
No ethics review needed.
Please see above
1. The paper introduces a novel perspective by integrating functionality with geometric and visual plausibility, significantly advancing the field of text-to-3D shape generation.
2. It achieves impressive results in generating 3D shapes, producing models that are both visually appealing and functionally viable compared with baseline methods.
3. The paper is well-written, presenting complex ideas with clarity and precision.
The paper introduces a novel text-to-3d approach Fun3D that integrates the laws of physics into 3D shapes. By analyzing solid mechanics, it addresses the limitations of existing methods that produce impractical shapes. Fun3D utilizes 3D diffusion models and a differentiable physics layer to optimize shapes according to physical laws. The paper also demonstrates its high-quality generation results with physical functionality.
1. The initial mesh can be generated not only from a 3D diffusion model but also by state-of-the-art (SOTA) models based on multi-view diffusion model and reconstruction framework, such as InstantMesh. It is recommended to use these advanced mesh generation models in place of Shape-E for better performance and versatility.
2. The paper includes too few baselines in comparison, limiting the comparative evaluation. It would be beneficial to add more baseline models to provide a broader and more robust comparison of the proposed method's effectiveness.
3.There should be a broader range of methods to assess the physical laws in the generated meshes, rather than limiting the evaluation solely to compression tests.
Yes
No ethics review needed.
1. Please address the concern about the weakness.
2. I am curious about the applicability of Fun3D to other types of objects where some of the parts are soft and some of the parts are solid. For example, in animals in Figure 6, the strength of the animal and the stress on them do not necessarily depend on the geometry we see on the outside. They are often more related to the structure of their bones and muscles. So why is Figure 6 shown or discussed in this paper, and why the proposed method can help animal generation to have better physical properties?
3. If some artist or architect wants to build something that is against the analysis of FEM, how do you balance the strength of the generated object and the design proposed by them?
1. By incorporating solid mechanics and FEM-inspired optimization into 3D generation, the paper advances text-to-3D methods beyond visual realism, aiming for physical feasibility, which is valuable for applications requiring structural integrity.
Differentiable 
2. The use of a neural network-based differentiable physics layer allows the system to be trained end-to-end, optimizing geometry while maintaining physics constraints.
3. The topic is important to real-world applications if we want to use a generative model to help produce solid objects.
The paper proposes Fun3D, a physics-informed text-to-3D generation method aimed at producing physically plausible 3D shapes based on text prompts. Existing text-to-3D models primarily focus on visual and geometric accuracy but lack physical realism, which limits practical applications. Fun3D addresses this gap by integrating physics, specifically solid mechanics, into the generative process. It uses a two-stage framework: an initial 3D shape is generated via 3D diffusion models and then optimized through a differentiable physics layer. This layer utilizes a mix of geometry and physics constraints, leveraging finite element method (FEM) data to improve stability and load-bearing capacity. Experiments demonstrate that Fun3D yields more physically robust shapes compared to baseline models like Diffusion-SDF, making it suitable for engineering and other real-world applications.
1. Limited Comparisons: The authors only compare their method to Diffusion-SDF, neglecting recent advancements like SDFusion (CVPR 2023) and LucidDreamer (CVPR 2024), which utilize different geometry extraction and physics-informed components. Including these would provide a fuller assessment of the method’s capabilities.

2. Evaluation Metric Bias: Physical strength is evaluated using FEM, which is also an integral component of Fun3D’s training. This could bias results in favor of the proposed model. Additional evaluation metrics, such as load capacity or material distribution uniformity, could provide a more unbiased assessment of physical properties.
Yes
N / A
No ethics review needed.
I cannot find important information about the evaluation. Table 1 gives quantitative metrics, but how many shapes are used in the comparison and how are they generated? How did you ensure that an unbiased and fair set of shapes are used in the comparison? Cherry-picking would greatly influence the quantitative comparison, so it would be good to know how cherry-picking was avoided. This is a major issue.

I am not sure about the conversion of SDF to a density field. This will create a transition of densities from the outside to the inside, but it is not clear how fast this transition is compared to the scale of the object. Can you please describe the parameters of this conversion or let me know where I can find them in the paper. This is a minor issue.
I can't decide the strengths of the method based on the current submission. The results do not seem to be competitive and a better evaluation and presentation of the results would be necessary to judge the strengths of the method.
The paper proposes a method to generate 3D shapes with increased physical plausibility. The pipeline is first initialized with the output of a traditional 3D generative model. The resulting shape is then optimized to have improved physical plausibility. In the examples, the method demonstrates how holes can be filled and weak support structures can made to be stronger.
I would think the positioning and naming of the paper should be discussed. The paper tries to position itself as 3D generation paper, but to me it seems to be a post-process that is not specific to 3D generation and not integrated with the 3D generation. Therefore, it can be applied to any 3D model. As a result, the comparisons are just a demonstration that the post-process improves upon the initial model, but not a comparison against other work. The Design Loss, the Geometry Constraint Loss, and the Volume Regularization Loss do not directly, but only indirectly consider the generative model. While the paper tries to set itself apart from other competing work by claiming that “All these methods do not need to learn physics feedback during shape learning. Unlike them, our method collaboratively learns geometry and physics during training.”. I cannot see how this is true. Please confirm if you want to claim that you train a generative model that uses physics during training of the generative model. Physics and generation in your approach does not seem to be integrated in an interesting manner. This is still fine for a good paper, but the results would need to be a lot stronger. Overall, the quality of the results is not impressive and they do not seem to be useful in the current stage.
Looks great, functions better
@inproceedings{
anonymous2024looks,
title={Looks Great, Functions Better: Physics Compliance Text-to-3D Shape Generation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=6SMeOas0JX},
note={under review}
}
Text-to-3D shape generation has shown great promise in generating novel 3D content based on given text prompts. However, existing generative methods mostly focus on geometric or visual plausibility while ignoring function for the generated 3D shapes. This greatly hinders the practicality of generated 3D shapes in real-world applications. In this work, we propose Fun3D, a physics driven functional text-to-3D shape generation method. By analyzing the solid mechanics of generated 3D shapes, we reveal that the 3D shapes generated by existing text-to-3D generation methods are impractical for real-world applications as the generated 3D shapes do not conform to the laws of physics. To this end, we leverage 3D diffusion models to provide 3D shape priors and design a data-driven differentiable physics layer to optimize 3D shape priors with solid mechanics. This allows us to optimize geometry efficiently and learn physics information about 3D shapes at the same time. Experimental results demonstrate that our method can consider both geometric plausibility and functional requirement, further bridging 3D virtual modeling and physical worlds.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
3D shape generation Functional 3D model Physics perception Differentiable physics layer Solid mechanics
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/485f4f11c3bf39c2443a11179a743b357c2f1bb0.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Looks Great, Functions Better: Physics Compliance Text-to-3D Shape Generation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission