Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting
articulated object modeling
Yes
No ethics review needed.
1. Have you tried to present the results when different movable parts are in different states?  Since this method models multiple movable parts with distinct part centers and dual quaternions, I believe it is possible to do that. It would be interesting to see those results.
1. The paper is well-organized and the overall writing is good.

2. The optimization speed is improved by using 3D Gaussian splitting.

3. Center-based part modeling and assignment allow the model to assign Gaussian to different movable and static parts. The use of part centers allows multi-movable parts compared to PARIS and does not require prior part segmentation knowledge compared to DTA. 

4. The self-guided motion priors with part dual-quaternions improve articulation learning with a further heuristic joint type prediction.

5. The experiments and ablation studies are extensive.
This paper introduces a part-aware articulated object reconstruction method with 3D Gaussian splatting. The method reconstructs the shape and articulation from two states of multi-view RGB-D images with a coarse-to-fine Gaussian initialization. A skinning-related center-based part modeling is designed to automatically assign Gaussians to statics and movable parts. The articulation type is learned through self-guided parameter learning.
1. Only state 0 and state 1 results are presented, the authors should present more intermediate states like Figure 8 in PARIS where they illustrated results on t=0.25, t=0.5,  and t=0.75.
Yes
No ethics review needed.
Please refer to the weaknesses section for further questions.
The key highlight of the paper is the innovative introduction of 3D Gaussian Splatting (GS) and superpoints, along with a coarse-to-fine Gaussian initialization strategy. Additionally, the method achieves state-of-the-art results on existing datasets, particularly excelling on a newly introduced dataset.
This paper presents ArtGS, an innovative method for reconstructing multi-part interactive objects. ArtGS utilizes 3D Gaussian Splatting combined with a coarse-to-fine standard Gaussian initialization strategy to achieve high-precision reconstruction of dynamic scenes.
The final visual renderings demonstrated in the paper primarily focus on cabinet-like objects, lacking validation on other categories. To better assess the generalizability of the method, I suggest adding renderings for a broader range of objects, such as chairs, scissors, and staplers, which have distinct motion components and structural differences. This would provide a more comprehensive evaluation of the method's applicability and accuracy across diverse objects.

Moreover, I have some doubts regarding the main contributions and novelty of this work. The success of this method may largely derive from elements such as the coarse-to-fine training strategy, joint optimization of static shape and articulation, the use of Gaussians instead of NeRF, and super-part rigging—all of which have been explored in previous research but are integrated here for this specific task. I would appreciate if the authors could further clarify their unique contributions.

Lastly, there is a lack of video examples to visually demonstrate the interaction of dynamic parts. Including video results would significantly help in understanding the method's advantages in complex dynamic scenarios.
Yes
No ethics review needed.
The below are some details of the approach or implementation that would benefit from clarification.

-	In Section 4.1:
	-	Once $ \mathcal{G}_{init}^c$ is initialized, are the Gaussians labeled as static/dynamics in binary or part ids?
	-	Since the refinement of $ \mathcal{G}_{coarse}^c$ only involves the static part, would the region of the movable part be very sparse? Does this sparsity remain for the optimization step later? Would it affect the reconstruction quality in the end?
	-	How many Gaussians are initialized in $ \mathcal{G}_{init}^c$?
-	In Section 4.2:
	-	What are the learnable centers $C_k$ useful for by intuition? What is $\mathbf{R}_k$ in $C_k$? Is $\mathbf{R}_k$ related to the articulation for the specific part?
	-	In equation 6, the variables $X_i^k$,  $\mu_i^c$, and $\mathbf{W}_\Delta$ are not defined.
	-	In lines 256-257, it is unclear how the distance matrix is related to part assignment. It is also unclear why identifying the sharp boundaries is the unique challenge in this case.
	-	In lines 258-260, why introducing $\mathbf{W}_\Delta$ is helpful in tackle the challenge?
-	In Section 4.3:
	-	In equation 7, $M_{ik}$ is undefined.
	-	In equation 8, $\mathbf{r}_i^c$ is undefined.

The below are some missing references for citation.
- [1] Yan, Zihao, et al. "RPM-Net: recurrent prediction of motion and parts from point cloud." arXiv preprint arXiv:2006.14865 (2020).
- [2] Jain, Ajinkya, et al. "Screwnet: Category-independent articulation model estimation from depth images using screw theory." 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021.
- [3] Lei, Jiahui, et al. "Nap: Neural 3d articulated object prior." Advances in Neural Information Processing Systems 36 (2023): 31878-31894.
- [4] Liu, Jiayi, Manolis Savva, and Ali Mahdavi-Amiri. "Survey on Modeling of Human-made Articulated Objects." arXiv preprint arXiv:2403.14937 (2024).
-	This work is contributing to an increasingly important research problem of building digital twins for arbitrary articulated objects with complex structure.
-	The experiments are showing promising results overall.
-	The observation and analysis of the experiment results are well discussed.
-	This work is well contextualized in the first two sections.
- This paper works on the task of articulated object reconstruction from multi-view RGB-D images.
- This work proposes to leverage 3D Gaussian as the underline representation to reconstruct articulated objects with multiple parts in a self-supervised way.
- Along with the representation, several strategies are proposed to improve the reconstruction accuracy, such as a coarse-to-fine initialization for part geometry and skinning-inspired dynamics modeling for articulation estimation.
- Experiments demonstrate the effectiveness and robustness of the proposed method.
-	**The writing of the Method section should be improved**. The quality of the writing overall is great, but the method section requires further improvement to articulate the details of the approach, especially for the logic of the presentation and usage of mathematical symbols. The current form hinders the understanding of the proposed method. Please see more details for clarification in the `Question` section.
-	**The experiment evaluation needs a bit more care**. 
	-	There are at least 17 highlighting of numbers in `Table 1` is misplaced, e.g., PARIS*[Axis Ang, Real Objects-Fridge] in the second row should be the second best.
	-	The “Axis Pos” reported in `Table 5` for synthetic objects is almost no difference given the limited significant figures. It would be more meaningful to show the metrics with more significant figures or report the numbers by timing 10 or even 100. 
-	**More qualitative results**. The presentation of the experiment would benefit from showing more qualitative results (in different complexity) that compares with the baseline methods, which can visually demonstrate the gap revealed in the metric evaluation.
-	**More discussion of the failure cases and limitations**. For the completeness of the paper, the failure cases and discussion of the limitation accordingly should be incorporated. There are assumptions that are not explicitly mentioned in the paper, e.g., the number of parts is given, the two states of the observation are pre-aligned in the world/object coordinate system. Maybe there are other assumptions that can be discussed, e.g., whether the articulated parts are required to be opened to expose visibility as much as possible.
Yes
Not appliable.
No ethics review needed.
- What is the comparison baseline for w/o Cano. init? Does it work if we just take one-state GS as initialization?  I think it should be ok as two-state is only w.r.t. some partial rigid transformations. 
- Can you explain the gap between obvious improvement in articulation parameters estimation and minor improvement in overall, geometry modeling?
+ The writing of the paper is easy to follow. 
+ The motivation of the paper is reasonable. By combining explicit 3D Gaussian modeling with proper articulation modeling, the method enables better articulation parameter estimation and object reconstruction.
+ The design for initialization and part segmentation based on different states and object parts is reasonable. 
+ Extensive experiments are conducted to evaluate the effectiveness of the method presented.
This paper presents a novel method for interactable articulated object modeling. The method leverages the skinning-like motion priors between object parts and the correspondences between two-state objects to model part-aware articulated objects. Results show that the results outperform existing works in both well-used benchmarks and datasets created by the author containing complex multi-part objects.
- The method is designed for two-state objects. It is uncertain how this can be applied to object captures with more states. Moreover, the number of parts is set manually as hyperparameters making it difficult to scale to different objects.
- It seems the method combines existing designs to achieve 3DGS-based articulated modeling. Though it is not super interesting, the overall pipeline makes sense. However, I am not sure what is the argument for using 3DGS compared to NeRF and MVS point clouds.  
- The authors only compare the method with DTA for visualization and with DTA and PARIS on DETA-Multi and ArtGS-Multi. Is it possible to compare with more methods? For example, some NeRF-based methods. Some related suggestions: 
 [1] Real2Code: Reconstruct Articulated Objects via Code Generation; [2] Articulated Object Neural Radiance Field; 
- More visualization results are essential for understanding the method to complement Fig. 4. For example, how well does the matching process and refinement work? I think these visualizations are vital for multi-stage methods. 
- I am not sure why perception-based metrics are not reported for evaluations, e.g., PNSR and LPIPS for 3DGS evaluation. 
- Though the paper is motivated by interactive articulation objects, I did not see related evaluation and applications for it. 
- I did not see discuss about failure cases and limitations.
@inproceedings{
anonymous2024building,
title={Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=ix2yRWarPn},
note={under review}
}
Building interactable replicas of articulated objects is a key challenge in computer vision. Existing methods often fail to effectively integrate information across different object states, limiting the accuracy of part-mesh reconstruction and part dynamics modeling, particularly for complex multi-part articulated objects. We introduce ArtGS, a novel approach that leverages 3D Gaussians as a flexible and efficient representation to address these issues. Our method incorporates canonical Gaussians with coarse-to-fine initialization and updates for aligning articulated part information across different object states, and employs a skinning-inspired part dynamics modeling module to improve both part-mesh reconstruction and articulation learning. Extensive experiments on both synthetic and real-world datasets, including a new benchmark for complex multi-part objects, demonstrate that ArtGS achieves state-of-the-art performance in joint parameter estimation and part mesh reconstruction. Our approach significantly improves reconstruction quality and efficiency, especially for multi-part articulated objects. Additionally, we provide comprehensive analyses of our design choices, validating the effectiveness of each component to highlight potential areas for future improvement.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
articulated object modeling
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/f7bbaea2377636ba83deda35599e573953662f3b.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission