TopoGaussian: Inferring Internal Topology Structures from Visual Clues
Gaussian Splatting;Differential Simulation;Topology Optimization;Neural Implicit Surface
Yes
No ethics review needed.
- The authors claim one of the applications is in 3D printing; since I lack any knowledge of 3D printing, I am unsure about its weaknesses. But to enhance my understanding, why do we require the internal topology to be known for this? Shouldn't having the surface not be enough?
- Please fix the typo in line 175 "point clout"
- Please keep writing style consistent for example, line 234, "point-cloud"
- Would it be possible again to summarize for me what actually is the main goal and main application of this work?
- Not so much a question but rather a comment regarding Abstract Style (improvement/suggestion): Having this kind of structure usually improves understandability and readability: 1. What is the problem and why is it important? 2. What are the limitations of existing solutions? 3. What are the advantages of the proposed approach? 4. How does it work? 5. Summary of results
The author works on a novel approach to solving a relatively novel problem for modern 3D Computer Vision. While the idea of reconstruction of internal topologies is not novel to the best of my knowledge, I have not seen much work trying to solve this for 3D Gaussian Splatting or NeRF-based approaches.
The work itself is decently written and shows a great evaluation to validate the quality of their methodology.
The authors can propose a holistic pipeline that should make their work easily usable for users, with the authors giving a significant amount of design possibilities.
In general, I am more than in favor of this work's results and core concept being interesting.
The authors present a holistic approach to estimating the internal topology of objects from images. The work relies on the use of a particle-based differentiable simulator to estimate probable internal topology directly from the seen motion of an object. By this approach, the author is able to generate even real-world 3D printed versions with reasonable structure directly from a small video sequence.
A large issue in this work is the convoluted writing. While still quite understandable, this work packs a significant amount of results, ideas, and concepts from many different fields.
As such, while many parts of the simulation (core contribution) have been well explained, much information regarding the volumetric representation is missing. As such, to improve the work (and make it complete), I would suggest the author add more information in the appendix.
Another larger issue is the motivation. While in Computer Graphics/Computer Vision, the challenge of estimating internal topologies is quite interesting, the estimation part might be a larger issue for practical, real-world use cases. In many real-world applications that rely on internal topologies, it is quite important that exact information is given, as this cannot assured by your model. I am still quite unsure when this work will be usable in real-world applications.
Yes
N/A
No ethics review needed.
Please see my questions in the previous weakness section.
1. Novel combination of Gaussian splatting with physics-based optimization for internal structure inference.
2. Uses a mesh-free approach that avoids common issues with mesh processing; presents particle-based differentiable simulations that are compatible with three flexible topology representations, including particle, neural implicit surface, and quadratic surface.
3. Well-structured presentation with a clear pipeline overview.
The paper presents TopoGaussian, a pipeline for inferring the internal structure of opaque objects using only photos and videos as input. The pipeline works by first using Gaussian splatting on multi-view images to obtain a point cloud, then optimizing the internal topology structure through a differentiable physics simulator to match observed motion patterns.

The key contributions include a particle-based, mesh-free pipeline that combines Gaussian splatting with a differentiable physics simulator; three flexible topology representation options: particle-based, neural implicit surface, and quadratic surface; a particle-based differentiable simulator supporting both rigid and soft objects with different topology structures.
The reviewer appreciates the authors' effort in building a particle-based pipeline to find a physically plausible internal topology structure. As the authors have also mentioned in the paper, this task is relatively new, and there are fewer baselines to compare with (at least other baselines do not use point-based representation). I have several concerns about the measuring metrics and their validation to support the claims from the authors:

1. Optimization Loss: This measures the difference between simulated motion and reference motion, and it directly indicates whether the internal topology structure is physically plausible. However, in Figure 3, the current method does not achieve the lowest loss among baselines in multiple test samples.
2. Comparison Implementation: When exporting the mesh from other baselines and chaining it into the rest of the pipeline in this paper, how can the mesh-based representation be made compatible with the rest of the system that is particle-based?
3. Time / Smoothness: It is unclear whether this improvement comes from the GS representation itself or from the authors' method. The reviewer encourages the authors to elaborate more on this or provide ablation studies to explain that the improvement comes from the proposed method itself.
4. Inner Structure: Can the authors provide the reference ground truth for the inner structures when making comparisons with other baselines? The reviewer understands that, in practice, the inner structures are hard to acquire, but in synthetic data, it is practical to obtain the ground truth of inner structures.

Other question:

How is the decision variable applied to the point cloud representation to obtain a continuous indicator function from the point cloud? (Line. 193)
Yes
/
No ethics review needed.
See weakness
1. Efficient and High-Quality Reconstruction: The particle-based approach of TopoGaussian achieves efficient reconstructions, with the authors reporting a significant speedup (5.26x faster) and superior boundary reconstruction quality (2.33x improvement) compared to mesh-based methods like PGSR and Gaussian Surfels.
2. Annotation-Free and Flexible: The pipeline’s independence from intrusive sensors or annotation requirements makes it practical and applicable in fields like robotics and manufacturing. Additionally, the three topology representation options offer flexibility based on application needs, from rigid to soft-body simulation.
3. Simplicity and Smoothness of Output: By eliminating the need for mesh processing, TopoGaussian produces smoother outputs conducive to 3D printing and manufacturing.
The paper introduces TopoGaussian, a particle-based approach that uses visual clues from photos and videos to infer internal topology structures of opaque objects. Key contributions include:
1. Mesh-Free Topology Inference: TopoGaussian combines Gaussian splatting and a particle-based differentiable simulator to infer interior topology without requiring mesh-based representations, which traditionally entail extensive fixing and filling processes.
2. Flexible Topology Representations: The pipeline supports different topology representations (particle, neural implicit surface, and quadratic surface), allowing for optimization and simulation within a unified framework.
3. Experimental Validation: TopoGaussian is evaluated on synthetic and real-world tasks, showcasing its capability to generate 3D-printable reconstructions that exhibit high fidelity and reduced processing time compared to existing mesh-based methods.
1. Evaluation Limitation: The method optimizes the interior structure based on a single motion, which may lead to overfitting and an inaccurate reconstruction of the true internal structure. Since ground truth data for internal topology is unavailable, it is difficult to verify if the inferred structure is correct or merely adapted to the given motion. Although the authors acknowledge this limitation and propose alternative metrics, these may not fully reflect the true structure. To strengthen the evaluation, the authors could consider obtaining ground truth through simulation (maybe with physics simulation) or testing the inferred structure on new motion videos as a test set. If the predicted structure is accurate, it should exhibit consistent behavior across these unseen motions, providing stronger validation.
2. Potential Overfitting to Single Motion: The current approach optimizes the internal structure based on a single motion video, which may lead to overfitting and limit the model’s generalization capability. To improve the robustness and accuracy of the inferred structure, the authors could consider optimizing based on multiple motion videos. By incorporating a variety of motions, the resulting model may better capture the true internal structure and provide more reliable and generalizable results.
3. Limitation to Simple Material Compositions: The current framework supports only single-object, dual-material compositions, which may limit applications involving complex, heterogeneous materials or multi-object interactions. Future work could focus on extending support to more intricate material compositions.
Yes
No ethics review needed.
● Is there a unique solution for recovering the internal structure of an object based solely on visual observation? How can we determine if the recovered internal structure is reasonable?

● The article mentions that the volumetric shape generation stage is faster than mesh-based methods. Is this stage entirely consistent with the PhyGaussian?

● What is the relationship between the metrics used in the article—reconstruction quality and optimization loss? How is reconstruction quality defined? Does the "Smoothness" mentioned in Section 8.1 Metrics represent reconstruction quality? Why does the proposed method achieve similar optimization loss to the baseline, yet show significantly better reconstruction quality than the baseline?
● Extracting an object's internal topology through a visual solution sounds interesting and is a novel task. The feasibility of the solution is also demonstrated.

● The article constructs multiple synthetic data to validate the effectiveness of its scheme, and more importantly, it accomplishes multiple sets of validation experiments in real-world scenarios.
This paper proposed a particle-based pipeline for motion reconstruction with physically corrected internal topology based on visual inputs. Specifically, they used a vision-based reconstruction method to construct the point cloud and optimize the physical properties of each particle through a differentiable simulation. Through several rigid-body and soft-body motion tasks, they verified the validity of the approach.
● The main goal of the article is to build the internal structure of the object by visual motion, but there is no proper metric or GT comparison to measure the correctness of the internal structure recovery, it only compares the rendering results

● The comparative experiments are insufficient; the paper only compares optimization loss, reconstruction quality, and time with two mesh-based approaches. This fails to highlight the main contribution of the paper, which is the recovery of internal structures. Additionally, the visual comparison of internal structures does not fully reflect the accuracy of the internal structure recovery.

● The proposed method involves multiple steps, such as Gaussian Splatting, Volumetric Shape Generation, and Topology Optimization, etc., but lacks thorough validation for these steps. For example, there is no detailed analysis of the impact of filling quality on topology optimization or the robustness of the entire system to the visual input quality and movement amplitude.
@inproceedings{
anonymous2024topogaussian,
title={TopoGaussian: Inferring Internal Topology Structures from Visual Clues},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=B5PbOsJqt3},
note={under review}
}
We present TopoGaussian, a particle-based holistic pipeline for inferring the interior structure of an opaque object with easily accessible photos and videos as input. Traditional mesh-based approaches require tedious mesh filling and fixing process with high error rate, while output rough boundary surface. Our pipeline combines Gaussian splatting with a novel particle-based differentiable simulator for objects, without interference with mesh. Based on the gradients from this simulator, we provides flexible choice of topology representation for optimization, including particle, neural implicit surface, and quadratic surface. The resultant pipeline supports modeling, simulation, and optimization of the interior geometry in a heterogeneous object on a unified point-cloud representation. We present a synthetic dataset, and showcase the results of our pipeline on it. Moreover, we perform experiments on four real-world tasks, and further 3D print the output to validate the efficacy of our methods. Compared with existing mesh-based method, our pipeline is 5.26 times faster, with 2.33 times quality of reconstructed boundary in average. These results highlight the potential of our pipeline in 3D-vision, soft-robotics, and manufacturing applications.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Gaussian Splatting Differential Simulation Topology Optimization Neural Implicit Surface
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/9b1758395ac6925ac8c5e2a7bbe5bb31e6331d6a.pdf
learning on graphs and other geometries & topologies
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
TopoGaussian: Inferring Internal Topology Structures from Visual Clues
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission