Revealing the 3D Cosmic Web through Gravitationally Constrained Neural Fields
computational imaging;signal processing;inverse problems;astrophysics;cosmology;neural fields;machine learning for physical sciences
Yes
No ethics review needed.
-- Could the authors elaborate on the training protocols and how difficult it was to get stabilized training? From the writeup, it is not clear how easy it is to get the model to work.
-- The paper attacks a problem of quite some importance. While the machine learning used is standard, its combination with the underlying physics allows the authors to obtain a promising solution, although it is only tested on simulations (standard for this area of physics at the moment). 

-- The paper is well-written, well-structured and easy to follow. While I am not an expert in this specific application, I have worked in some adjacent applications. I think the results support the underlying proposal and motivation well, and are creative in combining standard ML tools with physics-based models.
***Late review: Apologies my review is in one day late. I will aim to engage as quickly as needed to makeup.***

This very interesting paper considers a very important application, although using some well-established tools in machine learning and computer vision. The paper focuses on the difficult weak-lensing regime, where deflections are too weak to be interpretable reliably for a single galaxy. The idea is that attempting to understand how weak lensing alters (or shears) images of galaxies in dense fields, one could reconstruct a density map of the underlying matter. This is a difficult problem as the matter density is usually analyzed in 2D: the measured shear map is used to recover a projected 2D estimate. The paper aims to solve the inverse problem i.e. to obtain a 3D reconstruction of the dark matter field from 2D images. From a modeling perspective, this becomes challenging as galaxies are observed from a single view. Further, unlensed shapes of known visible sources are themselves not fully understood, introducing a large amount of uncertainty. The main idea of the paper is to use the underlying physics of gravitational lensing to recover a continuous 3D field, with a focus on also capturing non-Gaussian features, which are not easy with traditional methods, which use strong priors. The idea is to use coordinate-based neural fields augmented with the underlying physics (clearly described in figures 1 and 2), with the total loss function described in equation 7. The experiments, over simulated data, are able to show that the method has promise in both reconstructing the 3D matter field and also non-Gaussian features.
-- More of a question: Could the authors elaborate in the paper why are strong Gaussian priors usually made? This will not be obvious to a standard ML audience. I see that it could make sense from the perspective of filtering. But I am not sure I understand why this should drop out from the physics models. I would expect the features to be highly non-Gaussian. I find this confusing whenever I venture to look at papers in this area.
Yes
No ethics review needed.
* I would like to know some more details about the number of CPUs/GPUs used for training, memory requirements, and the training and inference time of the proposed method.

* I'm also curious about the next steps. After the 3D cosmic web estimation step is done, what are the next science questions that can be answered with the new results?
* The paper proposes a new method for an important physics problem.
* The paper is well-written; I enjoyed reading it.
* The physics background is exceptionally well-presented, and even non-experts can understand the main scientific ideas.
* The proposed method works better than the leading baseline.
The paper provides a new method for recovering the 3D cosmic web from galaxy weak lensing signal.
The proposed approach uses a coordinate-based neural field method with positional encoding. (Equation 6).
The reconstruction results are demonstrated in a series of experiments.
I think this is a great paper, but in my opinion, ICLR might not be the best venue for this.
The new machine learning contributions are quite limited: neural field models and positional encoding have been around for some time, and I didn't find significantly new machine learning contributions in the paper.

Nonetheless, the proposed algorithm works well, and the physics application, estimating the 3D cosmic web, is a very important problem.
Since the most significant contributions of the paper are in astrophysics, I think an astrophysics or cosmology journal would be a better venue for this paper.
Yes
No ethics review needed.
- it might be useful to related the various red-shift to a number of years, it is done for z=0 and z=2 but not for z=1. 
- The caption of Fig. 3 says: "results includes 12 lens planes from redshift z=0 to z=1", can the authors specify the precise value of the considered z ?
- In addition, the authors say that they average over 3 lens planes. Why ? to which values of z does that correspond ?
- The ground truth images are blurred with a factor that maximize the cross-correlation with the reconstructed picture. It seems a bit weird to do so, it might bias the reader to believe that a reconstruction is better than what it should be.
- The cross-correlation between the reconstructed signal and the true one seems quite small... while we can agree that it is even worse using the Wiener filter, I have the impression that it is working very-well.
The authors apply ML techniques to a dataset that is different from the usual benchmarks.
The article considers the task of reconstructing a 3D map of the universe from weak gravitational lensing obtained in 2D signals. The methods consider first to encode the overdensities using a neural field based on the spherical coordinate and its Fourier decomposition.  The target density field is used for two purposes: (i) to match the power spectrum of the dataset under consideration and (ii) to match the shear observed (after using the transformation from overdensity to shear) in the data. The method is tested on simulated data mimicking the instrumental observations from an N-body simulation. The results show that the method seems better than using Wiener filter for the same task.
My overall impression is that we have just an application of ML methods to a cosmological problem, which neither
- make relevant improvement on the side of ML (unless I'm mistaking, it takes out-of-the-box methods)
- make a strong advance in the considered field
I would like the authors to comment on how their work is particularly timely. I'm also thinking that the Wiener filter is not very modern. By quickly looking on the web, we can find more elaborated techniques:
 * sparsity prior: https://www.aanda.org/articles/aa/pdf/2021/05/aa39451-20.pdf, https://arxiv.org/pdf/1801.08945
 * wavelet: https://www.aanda.org/articles/aa/pdf/2006/21/aa2997-05.pdf
And I can imagine there are many others. This work does not provide any comparison to these methods.

The last section that investigate non-Gaussian structure is a bit of a mystery to me. Does the application of the method to MNIST (whatever it means) should be considered of a test of something precise ? In addition,
Yes
No ethics review needed.
- The authors use an ensembling approach, taking the median to get a point estimate on the reconstructed mass map. Much of the ongoing literature/effort is focused on _probabilistic_ approaches to mass mapping. The authors mention in a footnote that ensembling can lead to good predictive uncertainties -- can it be expected that this can be used to yield a calibrated distribution over plausible mass maps? If not, are there alternative approaches to this incorporating neural fields, e.g. variational or Bayesian approaches?
- What is the role of bandwidth $L$ and the induced spectral bias in the study? How does the induced prior compare with that assumed in traditional and model ML-based approaches?
- **Novel approach and good technical execution:** Using an implicit representation to model the underlying dark matter field along with a differentiable forward model is a very neat but challenging idea, and the authors set it up and executed it very well. 
- **Paper straddling AI and fundamental physics:** This is a bit of a meta point, but AI + physics (in particular a sub-field like cosmology) is not particularly mainstream at AI conference so far, in contrast to e.g. AI + bio. This makes presenting a paper like this while being faithful to the domain science particularly challenging, and the authors do a good job in this respect.
- **Simulation realism:** The authors use a realistic set of tools from the cosmology literature, e.g. JaxPM in the differentiable forward model, making it a contribution beyond a simple proof-of-principle. 
- **Baseline comparisons:** Comparisons with baselines are well-presented, and highlight the advantage of the method in particular regimes (e.g. in recovering non-Gaussian structures).
The paper presents a method for reconstructing the underlying 3D dark matter distribution from weak gravitational lensing observations (specifically, shape distortions of galaxies). This is an important problem in cosmology, and is of contemporary relevance given the number of astronomical surveys which will be measuring galaxy shapes. The overall approach is to model the dark matter distribution implicitly through a neural network (NeRF-like or implicit neural representation), and use a differentiable forward model that models the observed shear field. The authors compare the method to a few traditional approaches for mass mapping, finding it to especially perform well in reconstructing non-Gaussian structures.
- **Context in current literature and state of the field:** There exists substantial literature in neural approaches to mass mapping; e.g., https://arxiv.org/abs/2201.05561. While the authors compare to traditional approaches like Kaiser-Squires (KS) and Wiener filter, they do not contextualize their work in the more recent ML-based approaches to weak lensing mass mapping. Another example is https://arxiv.org/abs/2206.14820, which uses an implicit neural representation to perform *strong* lensing reconstruction; high-level comparison with existing literature could be improved.
- **Hyperparameter choices:** The authors mention specific hyperparameter choices, e.g. bandwidth of positional encodings L=2 and 5, without further justification. This should be expected to have a significant impact on the results, as it controls the spectral biases of the implicit representation, and should be further expanded upon, possibly including ablations.
- **Role of induced prior:** Traditional approaches typically have a regularization mechanism, often explicit, which allows for mass reconstruction (since the inverse problem is fundamentally ill-posed). In this study, the authors mention that "neural fields have been shown to provide a good implicit prior"; while true in the real-world setting of scene reconstruction, scientific problems are inherently different in nature, and it is not clear a-priori whether the implicit prior induced by the neural field is a good one. The authors approach this empirically, however further understanding of the role of induced priors is necessary for downstream science from the mass maps.
@inproceedings{
anonymous2024revealing,
title={Revealing the 3D Cosmic Web through Gravitationally Constrained Neural Fields},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=Ax0i933gtp},
note={under review}
}
Weak gravitational lensing is the slight distortion of galaxy shapes caused primarily by the gravitational effects of dark matter in the universe. In our work, we seek to invert the weak lensing signal from 2D telescope images to reconstruct a 3D map of the universe’s dark matter field. While inversion typically yeilds a 2D projection of the dark matter field, accurate 3D maps of the dark matter distribution are essential for localizing structures of interest and testing theories of our universe. However, 3D inversion poses signficant challenges. First, unlike standard 3D reconstruction that relies on multiple viewpoints, in this case, images are only observed from a single viewpoint. This challenge can be partially addressed by observing how galaxy emitters throughout the volume are lensed. However, this leads to the second challenge: the shapes and exact locations of unlensed galaxies are unknown, and can only be estimated with a very large degree of uncertainty. This introduces an overwhelming amount of noise which nearly drowns out the lensing signal completely. Previous approaches tackle this by imposing strong assumptions about the structures in the volume.  We instead propose a methodology using a gravitationally-constrained neural field to flexibly model the continuous matter distribution. We take an analysis-by-synthesis approach, optimizing the weights of the neural network through a fully differentiable physical forward model to reproduce the lensing signal present in image measurements. We showcase our method on simulations, including realistic simulated measurements of dark matter distributions that mimic data from upcoming telescope surveys. Our results show that our method can not only outperform previous methods, but importantly is also able to recover potentially surprising dark matter structures.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
computational imaging signal processing inverse problems astrophysics cosmology neural fields machine learning for physical sciences
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/bc1ef31a10e174c24a314624c25ef2803b1b3c62.pdf
applications to physical sciences (physics, chemistry, biology, etc.)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Revealing the 3D Cosmic Web through Gravitationally Constrained Neural Fields
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission