GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting
3D virtual try-on;3D Gaussian Splatting;diffusion model
Yes
No ethics review needed.
From line 62-68, is the pink stuff on the rightmost edited images because of the artifacts of the proposed method? 

Personally, I think this paper is a good follow-up on 2D virtual try-on method and will be beneficial to this research area. Extensive experiments and comparisons with state-of-the-art techniques demonstrate the superiority of GS-VTON in terms of realism and multi-view consistency. This is validated not only through quantitative metrics but also through user studies. Therefore,  I don't have any major concerns.
+ Effectively bridges the gap between 2D VTON and 3D applications by incorporating 3D Gaussian Splatting, which ensures consistency across multi-view images.
+ Uses a personalized diffusion model with LoRA fine-tuning, improving adaptability and customization for different subjects and garments.
+ Presents a new benchmark, 3D-VTONBench, which is an important addition for the comprehensive evaluation of 3D VTON performance.
They also demonstrates superior performance over existing methods, particularly in areas of realism, garment detail accuracy, and editing consistency.
This paper introduces a novel approach for achieving 3D virtual try-on (VTON) that addresses current limitations in consistency and spatial relationships when extending 2D VTON methods to 3D. The method, GS-VTON, leverages 3D Gaussian Splatting (3DGS) as a 3D representation framework, combined with personalized diffusion model adaptation using LoRA (Low-Rank Adaptation) fine-tuning. This allows for multi-view image editing with consistency across different viewpoints and high-quality geometric and texture fidelity. Additionally, the paper introduces a benchmark, 3D-VTONBench, to support quantitative and qualitative evaluations for 3D VTON methods. The experiments show that GS-VTON outperforms state-of-the-art techniques, establishing a new benchmark for 3D VTON performance.
- The model is a very straightforward follow-up of 2D VTON models and inherits some biases from pre-trained 2D VTON models.
Yes
No ethics review needed.
1. Could the authors provide more details on how the $G_{src}$ point cloud was collected in Figure 2? Since RGB-D sensors or other 3D sensors are less accessible compared to standard cameras, this may limit the method's real-world applicability. A more practical scenario might involve capturing multi-view images with a camera, but in that case, the lack of camera pose information could limit the feasibility of training the 3D GS model.

2. Could the authors provide more information about the response processing in the user study? Was a crowdsourcing platform used for data collection?
1. GS-VTON is the first 3D virtual try-on method, showing more diverse real-world applications compared to 2D virtual try-on. It holds promising potential to transform online shopping and create positive social impact.
2. GS-VTON utilizes Reference-driven Image Editing and 3D Gaussian editing to ensure the try-on scene is consistent in both texture and geometry across multiple views. The design seems sound.
This study proposes GS-VTON, a novel and potentially first-of-its-kind 3D virtual try-on method, based on a conditional diffusion model and 3D Gaussian Splatting. Unlike traditional 2D virtual try-on methods, GS-VTON enables users to visualize how clothing would appear on their bodies from different viewing angles, making it particularly promising for VR/AR applications. Moreover, the proposed reference-driven image editing and persona-aware 3D Gaussian Splatting techniques improves multi-view consistency in the virtual try-on experience in 3D.
1) **Reference-driven Image Editing**: The authors propose this method to ensure texture consistency across multi-view images by integrating attention features from a reference image. However, if the reference image has incorrect textures, it may negatively affect the consistency of subsequent images.

2) **Questionable Experimental Setting**: 
   - All benchmark methods use text as input, while GS-VTON uses an image as a prompt. However, the user study criterion of clothing image similarity may not be ideal for comparing these approaches.
   - Though the authors provide qualitative examples comparing GS-VTON with a baseline 2D VTON method, including this baseline in the user study for comprehensive quantitative analysis is essential. Most of the compared methods were not natively designed for virtual try-on applications, raising concerns about experimental fairness.

3) **Limited View Coverage**: GS-VTON mainly shows the front of the clothing, without displaying how the back of the body would appear.

4) **Pipeline Presentation Issue**: Figure 2 shows Reference-driven Image Editing as the first step, followed by the Personalized Diffusion Model via LoRA Fine-tuning. However, the main text introduces these components in reverse order, which caused some confusion initially.
Yes
None
No ethics review needed.
* In line 110, does LoRA "extend its learned distribution"? Are there any citations?
* I need more details about ControlNet. Is it from the original repository?
* The paper presents a groundbreaking method for 3D virtual try-on by extending pre-trained 2D VTON models to 3D using 3DGS, addressing the challenge of cross-view consistency and spatial relationships in 3D scenes.
* The establishment of the 3D-VTONBench dataset is a valuable resource for the research community, facilitating more comprehensive evaluations and fostering further advancements in 3D VTON.
* The method demonstrates superior performance over existing techniques.
The paper introduces a novel image-prompted 3D virtual try-on method that leverages 3D Gaussian Splatting for fine-grained editing of human garments within a 3D scene. The authors propose a personalized diffusion model adapted via LoRA fine-tuning to incorporate personalized information into pre-trained 2D VTON models. They also introduce a persona-aware 3DGS editing framework to maintain consistent cross-view appearance and high-quality 3D geometry. The paper establishes a new benchmark, 3D-VTONBench, for comprehensive 3D VTON evaluations and demonstrates through extensive experiments that the proposed GS-VTON method outperforms existing techniques in terms of fidelity and editing capabilities.
* In some cases, such as the first row in Fig. 1, there are noticeable artifacts on the sleeves and edges of the garments.

* The statements about the effects of persona-aware 3DGS editing are inconsistent between the abstract and introduction. The abstract states "maintain consistent cross-view appearance," while the introduction says "enhancing multi-view consistency."
* What are the differences or advantages of your methods compared to RelFill in "Personalized Diffusion Model via LoRA fine-tuning"? Some experiments may be needed to demonstrate this.
* The hyperparameter in persona-aware 3DGS editing seems tricky.
* To demonstrate the ability to maintain 3D consistency, the following works should be discussed.

  * Geometry-Aware Score Distillation via 3D Consistent Noising and Gradient Consistency Modeling
  * MaTe3D: Mask-guided Text-based 3D-aware Portrait Editing
  * ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusion
* Need more detailed descriptions of the proposed benchmark.
Yes
No ethics review needed.
1. It is better to discuss the reproducibility.
1. The paper is well-written and easy to understand.
2. This paper introduces a new perspective on persona-aware editing which effectively improves the performance of the 3D VTON task.
3. The limitations of the proposed method are well-discussed.
This paper introduces an image-prompted 3D VTON method (dubbed GS-VTON) which, by leveraging 3D Gaussian Splatting (3DGS) as the 3D representation, enables the transfer of pre-trained knowledge from 2D VTON models to 3D while improving cross-view consistency. Specifically, they propose a personalized diffusion model that utilizes low-rank adaptation (LoRA) fine-tuning to incorporate personalized information into pre-trained 2D VTON models. Moreover, they introduce a reference-driven image editing approach that enables the simultaneous editing of multi-view images while ensuring consistency. Furthermore, they propose a persona-aware 3DGS editing framework to facilitate effective editing while maintaining consistent cross-view appearance and high-quality 3D geometry. Additionally, they proposed a new 3D VTON benchmark, 3D-VTONBench, which facilitates comprehensive qualitative and quantitative 3D VTON evaluations. The experiments demonstrate the superior fidelity and advanced editing capabilities of GS-VTON.
1. There are concerns about the contribution of the proposed module, as it mainly adopts LoRA and a pre-trained diffusion model. For example, the X_train is not explained clearly in Line 339. Does it only refer to the results of the pre-trained IDM-VTON model?
2. I noticed that the paper does not provide evaluation metrics (e.g., LPIPS, FID). Including these metrics would improve the evaluation of the GS-VTON method.
3. There are some writing errors: a) Line 31: "GS-VTONhas" is missing a space. b) Missing commas in Functions 7 and 10.
a versatile pipeline for editing the garments of 3D human subjects with image prompts
@inproceedings{
anonymous2024gsvton,
title={{GS}-{VTON}: Controllable 3D Virtual Try-on with Gaussian Splatting},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=8eenzfwKqU},
note={under review}
}
Diffusion-based 2D virtual try-on (VTON) techniques have recently demonstrated strong performance, while the development of 3D VTON has largely lagged behind. Despite recent advances in text-guided 3D scene editing, integrating 2D VTON into these pipelines to achieve vivid 3D VTON remains challenging. The reasons are twofold. First, text prompts cannot provide sufficient details in describing clothing. Second, 2D VTON results generated from different viewpoints of the same 3D scene lack coherence and spatial relationships, hence frequently leading to appearance inconsistencies and geometric distortions. To resolve these problems, we introduce an image-prompted 3D VTON method (dubbed GS-VTON) which, by leveraging 3D Gaussian Splatting (3DGS) as the 3D representation, enables the transfer of pre-trained knowledge from 2D VTON models to 3D while improving cross-view consistency. **(1)** Specifically, we propose a personalized diffusion model that utilizes low-rank adaptation (LoRA) fine-tuning to incorporate personalized information into pre-trained 2D VTON models. To achieve effective LoRA training, we introduce a reference-driven image editing approach that enables the simultaneous editing of multi-view images while ensuring consistency. **(2)** Furthermore, we propose a persona-aware 3DGS editing framework to facilitate effective editing while maintaining consistent cross-view appearance and high-quality 3D geometry. **(3)** Additionally, we have established a new 3D VTON benchmark, 3D-VTONBench, which facilitates comprehensive qualitative and quantitative 3D VTON evaluations. Through extensive experiments and comparative analyses with existing methods, the proposed \OM has demonstrated superior fidelity and advanced editing capabilities, affirming its effectiveness for 3D VTON.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
3D virtual try-on 3D Gaussian Splatting diffusion model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/e33fdee32a0863f2be0811ecfec524037c7044ef.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/8873ff3b8d98dddb021c3c0479a6f73e67f16727.zip
GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission