Hi-Gaussian: Hierarchical Gaussians under Normalized Spherical Projection for Single-View 3D Reconstruction
Hierarchical Gaussian Sampling;Normalized Spherical Projection;Single-View 3D Reconstruction;Gaussian Splatting
Yes
No ethics review needed.
I mainly care about how much the paper depends on monocular depth estimation and generalization ability. Moreover, more qualitative results with arbitrary camera trajectory are also expected
1. Quantitatively, the paper outperforms all other methods in semantic KITTI dataset and BundleFusion dataset.
2. The method gives a clear rendering in unbounded street views.
The paper aims at the single-view 3D reconstruction which follows the Splatter Image [Stanislaw et al] and proposes two techniques to solve the limited FOV and occlusions. To give a broader view range, the paper converts the pixel coordinates to a normalized  Spherical Projection and then enlarges the FOV during the projection. For occlusion, the paper splats more gaussians biased from the leader gaussians that come from the per-pixel splat. The experiments show good performances in outdoor datasets.
1. The differences between the paper and the splatter image are using a pre-trained monocular depth network, changing the pixel coordinate representation as spherical coordinate and splatting more gaussians using a leader and follower way. These contributions are a little incremental and do not show good insights into this field.
2. I argue the performances may highly depend on the pre-trained monocular depth. Please show more experiments that the paper’s ability to tolerate the bad depth prediction in some cases where UniDepth does not work. Or show more results for the zero-show generalization ability on other images like tank and temple, LLFF dataset.
3.  The qualitative results are not enough. For example, only the zoom-in effect is shown and the view range in indoor room is also limited. Moreover, no videos are present to see the 3D consistency of reconstruction.
Yes
No ethics review needed.
1.Do you retrain the compared methods with your evalution cameras for a fair comparison? 

2.How you select the training and testing cameras?
The paper proposes to use normalized spherical projection for single view acne reconstruction and have some better results compared with baseline methods even though with blurry effects.
They propose a generalizable single-view 3D reconstruction method with Normalized Spherical Projection module for larger FOV reconstrcution. Some validation results show the effectiveness.
1.The approach to synthesizing unseen regions would benefit from a comparison with recent methods that utilize generative priors, particularly video diffusion models. Works like ViewCrafter and MotionCtrl demonstrate the effectiveness of such priors for single-view scene synthesis (Fig. 3 in ViewCrafter paper https://arxiv.org/pdf/2409.02048). Could the authors discuss how their method compares to these approaches, both theoretically and in terms of results?

2.The novel view demonstrations in Figure 3 primarily show viewpoint changes with substantial overlap with the input views (zoomed perspectives in the first two examples, and minimal FOV changes in the third and fourth). To better evaluate the method's capability in handling unseen regions, please consider including examples with more dramatic viewpoint changes."

3.The experimental comparison would be strengthened by including recent diffusion-based methods such as LucidDreamer, ViewCrafter, and MotionCtrl. This would help position the work within the current state-of-the-art in novel view synthesis.

4.The visual quality in Figure 3 shows some blurriness. Additionally, including rendered video results with camera trajectories would help better demonstrate the temporal consistency and overall quality of the synthesized views.

5.The robustness of the method could be better demonstrated through cross-dataset evaluation. This would help readers understand the generalization capabilities of the approach across different types of scenes and capture conditions.
Yes
No ethics review needed.
Please refer to the Weaknesses section. I combined Weaknesses and Questions since they are linked and need to be together for better clarity.
- The paper is well-written and clearly presented.
- The technical contributions are solid: Hierarchical Gaussian Sampling effectively models occluded areas—a critical challenge in single-view 3D reconstruction -- while the normalized spherical projection captures outside-FOV information, addressing limitations of the spherical projection used in existing techniques.
- The experiments, including both the main results and ablation studies on novel view synthesis, are comprehensive and demonstrate the advantages of the proposed approach.
- Figures 8 and 9 are particularly helpful to understand the impact of the proposed components.
The paper proposes a single-view 3D Gaussian Splatting (3DGS)-based 3D reconstruction method trained on a collection of posed images. This approach eliminates the need for expensive 3D ground-truth data for training and leverages the fast rendering capabilities of 3DGS. The method introduces two key contributions: a normalized spherical projection, which models information outside the field of view (FOV) while reducing the information compression typical of this projection, and hierarchical Gaussian sampling to effectively model occluded areas. The approach achieves SOTA performance in novel view synthesis in both outdoor and indoor datasets.
- The paper claims to address 3D reconstruction from a single image; however, it lacks any evaluation of 3D reconstruction. The paper should incorporate evaluations of 3D reconstruction and novel depth synthesis, similar to approaches like SceneRF, or include qualitative visualizations of 3D Gaussian distributions alongside depth maps, as demonstrated in pixelSplat, CVPR'24.

- The normalized spherical projection appears similar to the spherical projection technique proposed in SceneRF, with the primary difference being the normalization of x and y before projection. However, it's unclear how this normalization reduces spatial compression, as the paper suggests. What is the intuition behind this normalization?

- Figure 8 should be placed or at least referenced in Section 3.2 to illustrate the effect of normalization in spherical projection. Adding an explanation of the observed impact, specifically how normalization reduces black areas in the image, would help the readers understand more about the effect of the normalized spherical projection.

- Regarding Hierarchical Gaussian Sampling, since "leader" Gaussians are initialized using depth estimation, they are positioned on visible surfaces, with their distribution predominantly around these areas. It's unclear how "follower" Gaussians diverge from the "leader" Gaussian to cover occluded regions, given that they are sampled using the leader's distribution.

- Figure 7 would benefit from an additional column to highlight occluded scenery and areas outside the field of view in the novel view.

- The authors should clarify that the paper addresses single-view 3D reconstruction within a self-supervised setting, distinguishing it from monocular 3D reconstruction methods that often require 3D ground truth for supervision. I also think this is another selling point of the method and should be highlighted.

- On line 116, the authors mention choosing 3DGS over NeRF to achieve faster rendering speeds. Demonstrating the method's rendering speed relative to NeRF-based counterparts would provide valuable reference.
Yes
No ethics review needed.
Please respond to the three points made in the Weaknesses section.
The presentation is in general clear and easy to follow. The investigated problem of single-view reconstruction is of general interest to a broad audience.
The paper studies the problem of single-view 3D reconstruction in computer vision. Specifically,  a pre-trained monocular depth estimation model is used to provide initializations for the Gaussians, and an Encoder-Decoder network is employed to predict 3D Gaussian attributes from single-image inputs. A Normalized Spherical Projection module is proposed to expand the input FOV and enable reconstruction beyond the FOV, also a Hierarchical Gaussian Sampling strategy is proposed to reconstruct the occluded areas in the image. Experiments show that the proposed method performs favorably against the other competitors on scene-level single-view reconstruction.
Although the paper has presented better experimental results against existing methods, some major concerns remain:

1. Among all the competitors, SceneRF is the most relevant, which is also dedicated to single-view reconstruction at scene level. However, by comparison, this work is much more restricted as it requires a pretrained depth estimation model while SceneRF is self-supervised. This  also renders the comparison between SceneRF and this work unfair in Table 1, due to the use of pretrained model.

2. The Normalized Spherical Projection module seems to be heavily influenced by the Spherical U-Net module in SceneRF, which again weakens the contribution of this paper. 

3. The significance of single-view Gaussian Splatting does not entirely convince me. The occluded areas are unseen in a single view, and yet not hallucinated by any learned priors in the method as well. In a practical view, what is the significance of single-view Gaussian Splatting when occluded areas are visually blurry or even nonsense?
We propose Hi-Gaussian, which employs hierarchical 3D Gaussians under normalized spherical projection for efficient and generalizable single-view 3D reconstruction.
@inproceedings{
anonymous2024higaussian,
title={Hi-Gaussian: Hierarchical Gaussians under Normalized Spherical Projection for Single-View 3D Reconstruction},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=L3WnnnBRdu},
note={under review}
}
Single-view 3D reconstruction is a fundamental problem in computer vision, having a significant impact on downstream tasks such as Autonomous Driving, Virtual Reality and Augment Reality. However, existing single-view reconstruction methods are unable to reconstruct the regions outside the input field-of-view or the areas occluded by visible parts. In this paper, we propose Hi-Gaussian, which employs feed-forward 3D Gaussians for efficient and generalizable single-view 3D reconstruction. A Normalized Spherical Projection module is introduced following an Encoder-Decoder network in our model, assigning a larger range to the transformed spherical coordinates, which can enlarge the field of view during scene reconstruction. Besides, to reconstruct occluded regions behind the visible part, we introduce a novel Hierarchical Gaussian Sampling strategy, utilizing two layers of Gaussians to hierarchically represent 3D scenes. We first use a pre-trained monocular depth estimation model to provide depth initialization for $leader$ Gaussians, and then leverage the $leader$ Gaussians to estimate the distribution followed by $follower$ Gaussians. Extensive experiments show that our method outperforms other methods for scene reconstruction and novel view synthesis, on both outdoor and indoor datasets.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Hierarchical Gaussian Sampling Normalized Spherical Projection Single-View 3D Reconstruction Gaussian Splatting
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/3c29596a6da6d68c2d1c156d22ecc7b74d46999e.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/a77ad89d8c30cb3b8fa5727d3ec0aa3f24ee4035.zip
Hi-Gaussian: Hierarchical Gaussians under Normalized Spherical Projection for Single-View 3D Reconstruction
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission