studentSplat: Your Student Model Learns Single-view 3D Gaussian Splatting
3d reconstruction; 3d gaussian splatting; self-supervised learning
Thanks for finding our presentation clear. 

Weakness 1:

Thanks to the reviewer for pointing out the concurrent work. However, there is a fundamental difference between studentSplat and Flash3D. Flash3D requires a depth estimation model pre-trained using ground truth depth estimation which we have discussed similar methods in L126-139 and stated that they “either require[s] 3D supervision or only works at the object level” (L137-138). Technically, 3D Gaussian Splatting (3DGS) is a technique that reconstructs the 3D structure and color from multi-views and camera poses. If the 3D structures are reconstructed using ground truth annotation; only the color is reconstructed using the novel view, it should not be a 3DGS method, it is at most a method that uses 3D Gaussians as the representation. We are the first splatting method as we construct 3D Gaussians directly for multi-view supervisions instead of ground truth depth annotation. However, we recognize this ambiguity in definition and due to the presence of the concurrent work, we will modify our claim to “propose the first single-view 3D scene Gaussian splatting model that does not require relative camera poses during inference or ground truth 3D annotations.”

Weakness 2:

We disagree with the claim that “none of the list points are actually this paper's contributions”. 
First of all, our solution is unique and modular. In addition to extrapolating, our extrapolation design also “reduces distortion” (L92-93) as our design guide the novel-view reconstruction loss gradient from the missing context to the extrapolator but the in-context regions to the Rasterizer (L234-248). Additionally, our design automatically produces a context map (W) which allows the user to use a better extrapolation model during inference (L249-259) to fill the out-of-context region.

Secondly, instead of “distill[ing] the knowledge learned from multi-view images to single-view mode”, we “bridge the gap between multi-view 3D Gaussian splatting and self-supervised single-view depth estimation” (L95) which expands the use case of 3DGS to additional tasks and potentially can enable a new way of training single-view depth estimation model. For example, current methods usually obtain depth labels from ground truth (lidar sensor), sparse labels from SfM or disparity (if multi-view data is available). Using our proposed training method (studentSplat), we have another way to obtain dense pseudo depth labels. 

Finally, we did not propose a new applications, rather we connected 3DGS to different applications. To make “if depth/3DGS is available” happen, unless we have ground truth 3D annotations, the current solution is to use multi-view input (with camera pose) (Flash3D uses ground truth depth label in the form of pretrained depth estimation model). We want to emphasize that studentSplat produces depth and 3D Gaussians using only multi-view training data. For text-to-3d, expanding a text-to-image method to text-to-3D is impossible for multi-view input in a feed-forward manner (MVSplat or the alike) since it is impossible for the current text-to-image method to generate two or more images with only camera view change. Although there are method for each application, given multi-view training data (no depth label), studentSplat bridges single-view 3DGS, depth estimation, and text-to-3D.

Weakness 3:

We agree that distillation is commonly used and we did not propose a distillation method. Our contribution is to enable single-view 3DGS using only multi-view training data; distillation is a component. Even as a componet, “extrapolation module is simply learning from novel views” is not our approach. We have commented on our extrapolation approach above in Weakness 2; our extrapolation module is designed to perform extrapolation, produce context mask, and to guide gradient flow.

Weakness 4:

I agree that “single-view reconstruction is practically close to impossible in "estimation" tasks” for the missing region. Therefore, our method reconstructs the 3D structure of the in-context regions (evaluated by the depth estimation Table 3). The out-of-context regions are “generated” using the extrapolator. 
In the real applications, we have a lot of 2D contents like photos and paintings. As AR and VR devices are getting more accessible, we also need 3D contents. The easiest way is to convert 2D to 3D using the proposed studentSplat. Compared with others that use GT depth information, studentSplat enjoys a larger variety of data source, lower collection cost, and wider application. Finally, “extrapolating better than the generation model” is an open problem. However, our method has the possibility to achieve better extrapolation if the field later has a better solution; our method estimates the context map (W) which allow the user to apply different extrapolation methods or generation methods during inference whereas Flash3D or the methods alike does not have this advantage.
Clarify the contributions and reiterate the uniqueness of our method.
Thanks for the comments on the effectiveness of our method.

First of all, we want to clarify the difference between tasks.

Novel-view reconstruction: aims to reconstruct the novel-view instead of 3D structure.

Depth estimation: aims to estimate the 3D depth instead of novel-view.


Weakness 1:

We agree with the review that the 3D consistency in the extrapolated region is not assured. However, we never aimed to reconstruct the 3D structure of the extrapolated region. Similar, the teacher model (MVSplat) also did not assure the 3D consistency in the missing region. As stated in L226, the purpose of the extrapolator is to improve the novel-view reconstruction instead of performing 3D structure reconstruction in the missing region. Note that in the novel-view reconstruction task, we do not require reconstruction of 3D structure. 

The “3D Consistency” is ensured in the in-context region and we demonstrated it through self-supervised single-view depth estimation task (Table 3).


Weakness 2:

Thanks to the reviewer for pointing out the concurrent work. We agree that including it will improve the overall presentation of our paper. However, there is a fundamental difference between studentSplat and Flash3D. Flash3D requires a depth estimation model pre-trained using ground truth depth estimation which we have discussed similar methods in L126-139 and stated that they “either require[s] 3D supervision or only works at the object level” (L137-138). In the application setting, this difference will rule out Flash3D as a self-supervised depth estimation method. Technically, 3D Gaussian Splatting (3DGS) is a technique that reconstructs the 3D structure and color from multi-views and camera poses. If the 3D structures are reconstructed using ground truth annotation; only the color is reconstructed using the novel view, it should not be a 3DGS method, it is at most a method that uses 3D Gaussians as the representation. However, we recognize this ambiguity in the defination and due to the presence of the concurrent work, we will modify our claim to “propose the first single-view 3D scene Gaussian splatting model that does not require relative camera poses during inference or ground truth 3D annotations.”

Question 1:

It is about 2 times faster to train and test than splatterImage. However, we are proposing the training method and different models can be used directly with our proposed training method. Therefore, we simply used a popular model design.

Question 2:

MI-GAN was fine-tuned during training. We selected MI-GAN only because of its efficiency. As discussed in L249-259, we do not aim to produce perfect extrapolation results. As long as we can generate a mask (W) of where the in-context and out-of-context regions are, we can always use a more powerful extrapolation model during inference.

Question 3:

The proposed method for refining student output uses the multi-view teacher model to perform 3DGS on the output of the student model along with the single-view input image. Basically, we aim to also produce 3D Gaussians for the extrapolating region since the extrapolator only extrapolates in the 2D reconstructed view. As suggested by the reviewer, the teacher model can also be added after the student model during training to improve the performance while still requiring only one image during inference. However, it will not add too much to our existing training pipeline and should not provide additional insight compared to what we have in Appendix D.
Clarification on the difference between tasks, extrapolation, and our contributions. Answer the questions.
Thanks for the comments about our clear presentation and strong model performance.

Weakness 1:

First of all, we agree that the context and target are not far enough. We use the same views as proposed by previous works. Making the context view and target view more different will likely reduce the performance for all the methods evaluated but not affect the ranking of the methods. Thus, we followed previous works instead of proposing our own evaluation.  Additionally, the gap between the best performing model and the worst performing model (Table 1) is large enough to suggest that the evaluation is valid and simply copying the input as the target will produce bad results.

Secondly, to make sure there is 3D geometric structure is learned, the best way is to evaluate the geometric structure directly. Therefore, we evaluated the single-view depth estimation performance in Table 3 in addition to the novel-view reconstruction performance. Note that previous works are only evaluated on novel-view reconstruction.


Weakness 2:

Note that our model in the single-view depth estimation setting is purely self-supervised despite having the same architecture as Depth Anything. Therefore, comparing it against a large-scale supervised method is unfair. Additionally, we are not restricting the architecture. We only proposed the training method, the model architecture can be changed. Finally, being smoother compare to the ground truth is again due to the lack of ground truth supervision. But compare to the other self-supervised methods (GasMono, SpaltterImage, Figures 9 and 10), studentSplat is reasonably sharp.


Question 1:

First of all, we disagree with the author that the performance improvement over SplatterImage is “a little”, the improvements are at least 5% for all datasets. For novel-view reconstruction, studentSplat not only outperforms the splatterImage on the quantitative results (Table 1), it also has much less distortion (Figure 3, Appendix Figure 8). For single-view depth estimation, studentSplat is comparable to a self-supervised single-view depth estimation method (GasMono) while splatterImage is much worse (Table 3). Additionally, splatterImage has a lot of noise whereas studentSplat is much clearer (Appendix Figure 9).

Question 2:

Figure 5 shows the difference between the extrapolator and w/o extrapolator and appendix Figure 7 shows the weight for the original context and the extrapolating target (which indicating the extrapolating region).

Question 3:

Thanks for noticing the visualization error, we will correct it.

Question 4:

We will correct typos.
Clarify the contributions and answer the questions
Thank you for you comments about the strong performance and efficiency of our studentSplat.

Questions 1:

As stated in our contribution our method “does not require relative camera poses during inference” (line 90). We will still require camera poses (SfM) during training. It is impossible to train a novel-view reconstruction method using only an image dataset. Therefore, like all other novel-view reconstruction methods, our model needs to be trained on a multi-view dataset (either video or multi-view). We then need to perform SfM to obtain the camera poses for the training data. However, we do not need relative camera poses during inference as claimed. During inference, only one “fake” camera pose (any camera pose defined by the user) is needed for the entire testing dataset since we only need one input image. Therefore, no real camera pose is needed. To further support the claim of “mitigating the camera pose”, we performed the single-view depth estimation using the one input image and the same camera view (for the entire dataset) defined by the user (Table 3). 


Question 2:

There are both qualitative (Figure 5) and quantitative results (Table 4) of the performance of studentSplat w/o extrapolation in the ablation study.


Question 3:

First of all, in Table 4, we can already see that the performance of studentSplat without extrapolator (PSNR 21.38, SSIM 0.741, LPIPS 0.208) is significantly better than MVSplat (PSNR 17.73, SSIM 0.585, LPIPS 0.296). Secondly, MVSplat (one view) + extrapolator will require training as the extrapolator cannot work with a 3DGS method out-of-the-box. As studentSplat without extrapolator already outperform MVSplat by a large margin, adding extrapolation should not flip the result.
Clarify what the reviewer has missed from our paper and answer the questions.
Yes
There are no concerns.
No ethics review needed.
- As mentioned in the weakness section, training the student network in the same dataset with the teacher network cannot be claimed as fully mitigating the camera pose constraint. Can the model be trained in a new dataset the teacher has not been trained on? Or can the student network be trained with the rendered results of the teacher instead of ground truth images?
- To fully understand the performance of StudentSplat without the extrapolator, can the authors provide qualitative and quantitative results without the extrapolator?
- Extended from the previous question, I am concerned that the performance of studentSplat without the extrapolator is similar to the performance of MVSplat with Gaussians from one context images. As a result, can the authors show the performance of MVSplat (one view) + extrapolator?
- The overall paper is well-written with the architectural designs mentioned in detail making the readers easy to understand the training procedure and contributions of the work.
- Although the task being challenging, the proposed method shows strong performance, achieving state-of-the-art in multiple datasets.
- The proposed method is efficient in terms of the model parameters and the number of Gaussians compared to previous methods.
The paper introduces studentSplat, a single-view 3D Gaussian splatting (3DGS) aimed at scene-level reconstruction from a single image. Recognizing challenges in single-view reconstruction, such as scale ambiguity and context extrapolation, the authors propose a teacher-student architecture. Here, a multi-view teacher (pre-trained feed-forward Gaussian Splatting Model (e.g., MVSplat)) provides geometric supervision to a single-view student model during training, addressing scale ambiguity and encouraging geometrically valid reconstructions. Additionally, the model uses an extrapolation network to adaptively fill missing scene context through GANs, improving the novel-view reconstruction quality.
- Mitigating the use of camera poses : The authors mention that the camera pose of a single image can be defined as the identity matrix, mitigating the use of camera poses of multi-view images. However, during the teacher-student geometric supervision, as MVSplat[1] has been trained on both RealEstate10K and ACID using SfM Camera poses, this supervision guides studentSplat to learn this SfM Camera Pose scales which enables the photometric loss of $L_{photo}$ with a specific relative camera pose $[R|t]$. As a result, I strongly believe that the current training scheme cannot be claimed as mitigating the camera pose.
- Performance contribution of extrapolator : As the evaluation is currently done with 2 novel views outside and one novel view inside the context frustums, I agree that the extrapolation performance of StudentSplat is superior than other methods and the teacher MVSplat. However there is no qualitative or quantitative results of the performance of studentSplat without the extrapolator which makes it hard to fairly compare with other methods.
Yes
No ethics review needed.
L90: The claim ‘Propose the first single-view 3D scene Gaussian splatting model’ is not convincing. Though SplatterImage (CVPR 2024) mainly focused on object-level scenes in the paper, the results in Table. 1 show that SplatterImage perform relatively well on large scenes (a little worse than the proposed method). 

Can you show some visualization of extrapolation with MI-GAN? 

L1009-1015: The visualization looks wrong. The input images and target image are from different scenes.

Typo: L237: compositing-> composition
The idea of distilling knowledge from multi-view model to single-view model is simple and makes sense. 

The paper is clearly written.

Studentsplat outperforms recent methods in single image rendering and monocular depth. 

The ablation study in main paper and supplementary is thorough.
This paper introduces studentsplat. Inspired by some recent feed-forward 3D Gaussian Splatting methods with multi-view or single-view inputs, studentsplat combines two settings and uses multi-view model (teacher) to improve the performance of single-view model (student). An extrapolation network (GAN) is used to complete missing scene context and thus facilitates training. The idea is straightforward and the performance is better than recent methods in single image rendering and monocular depth quality.
I felt the evaluation is not very convincing. In single image setting, it is not surprising that pixelsplat and mvsplat perform worse since they rely on feature matching across different views, which is unavailable with single image (or let’s say two same images with baseline=0). For the evaluation with single image, my concerns mainly come from Fig. 3, 4, 7, 8. In these images, we can find that the viewpoints of target views and input context view are similar (i.e. the baseline between input view and target view is small). In some cases, I think directly copying the input image as output would provide nice rendering metrics as well. Therefore, I think evaluation with large baseline between input context view and target view should be included. 

In Fig. 11, seems the rendered depth from studentsplat is usually oversmoothed. Additionally, since the authors also mentioned Depth Anything (the student model also has similar structure as Depth Anything, i.e. DINOv2 + DPT), comparison with Depth Anything on depth quality can be included.
Yes
No ethics review needed.
* Given the model's relatively low parameter count and the use of a lightweight generative model, does this method offer any advantages in terms of training time and inference speed compared to other single-view methods?
* Is the generative model MI-GAN fixed or fine-tuned during training? Could the authors provide more detailed experimental settings and analyses regarding MI-GAN?
* The authors propose a method for refining student output in Appendix D. Can this method be utilized during the training process of the student or teacher models as a form of self-supervised approach to enhance multi-view consistency?
* The approach of leveraging geometric priors from multi-view reconstruction methods to enhance single-view reconstruction is intriguing, and the authors have experimentally demonstrated significant improvements in the perspective extrapolator through multi-view distillation.
* The authors present a simple yet effective method for extrapolating when computing the novel view reconstruction loss.
* The paper is well-written and easy to follow.
The paper presents a pioneering method for single-view 3D Gaussian splatting, addressing challenges in scene reconstruction from single images. The authors introduce a teacher-student architecture, where a multi-view teacher model provides geometric supervision, mitigating scale ambiguity. Additionally, an extrapolation network enhances scene context completion, leading to high-quality reconstructions. The proposed method achieves state-of-the-art results in single-view novel-view reconstruction and demonstrates potential in self-supervised depth estimation, broadening the applicability of 3D Gaussian splatting models​.
* **3D Consistency**: In the unseen regions when extrapolating new views, the rendered results depend on the 2D generative model MI-GAN. Therefore, I am skeptical about the model's ability to generate extrapolated continuous new views with 3D consistency. I recommend that the authors supplement the discussion with relevant visual results or theoretical analyses.
* **Overclaim**: The authors state in the contributions section that they "propose the first single-view 3D scene Gaussian splatting model that does not require relative camera poses during inference." However, to my knowledge, there exists a single-view 3DGS-based method, Flash3d [1], which can achieve much of what this work does without needing camera poses. I recommend that the authors explicitly address how their method compares to Flash3D and explore the differences in their approach. Additionally, if feasible, including comparative experiments in their analysis would enhance the robustness of their claims.

[1] Szymanowicz S, Insafutdinov E, Zheng C, et al. Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image[J]. arXiv preprint arXiv:2406.04343, 2024.
Yes
No ethics review needed.
Please see the weaknesses above.
1. Clear writing
2. Experiments are conducted with large-scale datasets, and compared with existing SOTAs.
This paper proposes a single-view 3D scene reconstruction method. While single view reconstruction suffers from unknown scale and regions unobserved, the authors introduce two techniques. The first solution is introducing a teacher-student architecture where multi-view model acts as a teacher model to provide supervision to student model. The second solution is introducing an extrapolation network that helps to extrapolate. The method is evaluated on ACID and RE10k, and ablation studies are conducted to support author's design choices.
1. In page 1, line 38, why is it the first single-view 3D gaussian splatting method? I recall flash3D [1], the paper written by the same first author of the splatterimage,  has already publicly available since June 2024, which is hard to miss. I understand that the paper seems to be yet published to any conferences or journals, but this does not mean that the authors can simply ignore this already existing paper and claim author's paper as the first approach. I recommend the authors to cite this paper, as well as provide some comparisons to them. I find that their code implementations are also available. 

2. From the listed contributions in Line 90~96 at page 2, none of the list points are actually this paper's contributions. For example, the first approach for single-view 3D scene GS model is achieved by Flash3D, extrapolation issue is an apparent issue that comes with single view reconstruction, which many other works also address. Finally, It is questionable whether expanding the applications of 3D Gaussian splatting model is also one of the contributions. Since there have been many attempts trying to distill the knowledge learned from multi-view images to single-view model, and the applications the authors show are not really new. Single-view depth estimation is, NVS, Text-to-3D are all already existing applications if depth/3DGS is available. 

3. It is difficult for me to find technical contributions as well. Distillation from teacher to student is commonly adopted in the community and extrapolation module is simply learning from novel views, which have already been done in flash3d. 

4. Finally, I believe single-view reconstruction is practically close to impossible in "estimation" tasks, where 3D geometry dominates. It is possible in "generation" tasks, but since we never know what is beyond the single view observation, I personally believe this task is ambiguous in current state. Unless the authors provide convincing arguments that justify the need of this task and the feasibility of extrapolating better than the generation model, I doubt this work will deliever a valuable message to the community.


[1] Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image, arxiv'24
@inproceedings{
anonymous2024studentsplat,
title={studentSplat: Your Student Model Learns Single-view 3D Gaussian Splatting},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=fRXAQfHlmr},
note={under review}
}
Recent advance in feed-forward 3D Gaussian splatting has enable remarkable multi-view 3D scene reconstruction or single-view 3D object reconstruction but single-view 3D scene reconstruction remain under-explored due to inherited ambiguity in single-view. We present studentSplat, the first single-view 3D Gaussian splatting method for scene reconstruction. To overcome the scale ambiguity and extrapolation problems inherent in novel-view supervision from a single input, we introduce two techniques: 1) a teacher-student architecture where a multi-view teacher model provides geometric supervision to the single-view student during training, addressing scale ambiguity and encourage geometric validity; and 2) an extrapolation network that completes missing scene context, enabling high-quality extrapolation. Extensive experiments show studentSplat achieves state-of-the-art single-view novel-view reconstruction quality and comparable performance to multi-view methods at the scene level. Furthermore, studentSplat demonstrates competitive performance as a self-supervised single-view depth estimation method, highlighting its potential for general single-view 3D understanding tasks.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
3d reconstruction; 3d gaussian splatting; self-supervised learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/e9756f35a3313e11f3f5a226a8fb75ad52f7ab50.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
studentSplat: Your Student Model Learns Single-view 3D Gaussian Splatting
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission