Grounding Continuous Representations in Geometry: Equivariant Neural Fields
Geometric Deep Learning;Neural Fields;Equivariance;Representation Learning;Latent Point Clouds
Yes
No ethics review needed.
1. Does it make sense to compare against Inr2Array [1]?
2. Should NF2vec also be a baseline for the shape classification task (Table 2)?
3. Can Functa be trained with a cross-attention-based architecture, similar to that proposed for ENF?
4. For Functa baselines on downstream tasks such as classification, what was the architecture of the decoders used? For tasks involving generalization, 

[1]: Zhou, Allan, et al. "Neural functional transformers." Advances in neural information processing systems 36 (2024).
This paper creates a novel equivariant neural field based on the notion of steerability from equivariant networks, which has the advantages of weight-sharing, locality, and geometric interpretability.
This paper proposes equivariant conditional neural fields based on steerable networks. Architecture-wise, this paper proposes equivariant  cross-attention layers with Gaussian windowing as the basis of their Equivariant Neural Fields (ENF). The ENF is trained with a two-stage process: in the first stage, the ENF backbone takes in an input signal and outputs a latent point cloud of (pose, context) pairs. Downstream tasks can be accomplished by training a decoder which takes the latent point cloud as input. Experiments are performed on 2D image reconstruction and classification, 3D reconstruction, classification, and part-segmentation, flood map segmentation, and climate forecasting.
1. The original Functa paper uses a SIREN neural field architecture but this paper uses an attention-based neural network architecture. This seems like a potentially unfair comparison. 
2. Another weakness of this paper is that there is no way to decide ahead of time whether to train the latent point cloud using MAML or autodecoding. 
3. The only baseline is Functa for most experiments. Is it possible to use NF2vec in Table 2 and Inr2Array [1] for any of the experiments involving downstream tasks? For tasks involving generalization, 
4. ENF performs only comparably to to the baselines on part-segmentation (Table 3), and some experiments (Table 2) don't show the effectiveness of using equivariance.
Yes
No ethics review needed.
Please refer to the weakness section
***Clear and Professional Presentation***: The paper is well-written, structured effectively, and easy to follow. Its clear motivation, logical organization, and high-quality visualizations contribute to a polished and professional presentation, making the methodology accessible and engaging.

***Introduction of Equivariant Neural Fields Model***: The authors propose a novel model, Equivariant Neural Fields, which combines conditional neural fields with point cloud conditioning and equivariant decoding from latent space to field. This approach creatively integrates Neural Fields with equivariant models designed for point clouds, expanding on existing techniques. Additionally, the paper introduces specialized attention layers and engineering optimizations that enhance the model's efficiency, showcasing an innovative blend of established methods.

***Comprehensive Experimental Validation***: The method is rigorously tested across a wide range of use cases and downstream tasks spanning various domains. This extensive evaluation demonstrates the versatility and potential real-world applicability of the proposed approach, supporting its robustness and utility across diverse applications.
The paper introduces the Equivariant Neural Fields, a variant of conditional neural field that uses a geometry-informed cross-attention to condition the NeF using geometrical point cloud representation. The method was validated using a variety of applications, including classification, segmentation, forecasting, and reconstruction.
***High Time Complexity***: The proposed approach appears to be computationally intensive. It would be beneficial for the authors to compare the training time and memory usage of their method against a reference model, such as the Functa method, to provide a clearer assessment of its efficiency.

***Lack of Ablation Studies***: The paper would benefit from ablation studies to clarify the contributions of key components, such as Gaussian spatial windowing and the k-nearest neighbors (kNN) efficiency trick. These studies would help demonstrate how each element impacts the model’s training efficiency and overall performance.

***Suboptimal Segmentation Performance****: The segmentation results are weaker than those of traditional point cloud segmentation baselines. A deeper investigation and discussion of these performance differences would help in understanding and potentially addressing the gaps in segmentation accuracy.
Yes
No ethics review needed.
- I'm particularly curious about the use of these equivariant neural fields as a general backbone for any neural field based task? Are there any situations where it's not helpful to enforce equivariance especially for vision / PDE-based applications?
- Have you considered using this methodology in a generative context? I think the localized latent point clouds are a particularly interesting property that could lead to more structured creation.
- Did you study the sample efficiency of ENFs against other CNF methodologies in tasks such as classification? One would assume that enforcing equivariance should lead to a better sample efficiency throughout all truncations of the training dataset
- I'm curious about the computational cost of your experiments. Does it have a similar run time to the other baselines that were discussed?

Additionally, I believe there are a couple of typos that I may have spotted:
- In the abstract: faitfhully -> faithfulll
- Also, on line 103, posses needs to be possess?
- The paper introduces a novel and mathematically sound method to incorporating geometric structure to neural fields through the equivariant cross attention. The steerability property is well formulated with proven bi-invariant constraints.
- The experimental details demonstrate an advantage over methods that do not incorporate such geometry informed structure in the latent space. Additionally, the locality and weight-sharing properties discussed are clearly demonstrated.
- The paper is well-written providing clear background on the neural fields, and the motivation for the need for enforcing equivariance in neural fields. The diagrams are informative and highlight the key components of the methodology. Highlighing geometry attributes in Section 3 with a blue text color was particularly helpful in aiding understanding
The paper introduces a novel class of Conditional Neural Fields (CNFs) called Equivariant Neural Fields (ENFs) which aim to address the limitations of CNFs in tasks requiring geometric reasoning. The authors propose a geometry-informed cross-attention mechanism that conditions on a latent point cloud of features, enabling equivariant decoding from the latents to the field of interest. This approach possess a steerability property where transformations in the field and mirrored in the latent space. Further, this approach ensures that the cross-attention attention operators respond similarly regardless of pose allowing for weight sharing over similar local patterns leading to more efficient learning. These claims are backed with experiments that demonstrate the advantages posed by the formulation and show a clear advantage over the baselines that have a geometry-free latent space.
- While the motivation to compare against other CNF based approaches is clear, the methodology seems to be restricted to a discussion and comparison to the results reported in functa (Dupont et al.) and other CNF-based methods but do not provide a thorough comparison against other equivariant methods or other state of the art methods. Perhaps a comparison of ENFs against more comparisons would strengthen the paper.
Yes
No ethics review needed.
I would appreciate a response regarding the weaknesses and questions mentioned above.
The paper is well-written and organized, with a clearly defined method supported by formal definitions.

The proposed solution is simple and intuitive for enhancing CNFs with local equivariant features.

I appreciate the variety of dataset types used in the experiments.
The paper presents a method for conditioning a neural field using a set of $SE(n)$ equivariant local latents. The aim is to enhance downstream task performance by operating on the neural field’s learned latent representation, rather than on discrete samples from the continuous signal as in conventional approaches. It outlines the necessary conditions for an equivariant latent representation in neural fields and adapts a cross-attention architecture to support these conditions. The approach is evaluated across a wide variety of tasks.
Lack of motivation for using CNF latent encodings in downstream tasks

The paper does not explicitly discuss the motivation for using latent encodings of CNFs for downstream tasks. It seems that one advantage might be the ability to utilize more data for training since the reconstruction training stage does not require labeled data. This raises a follow-up question:
What benefit do latent features learned through continuous reconstruction (decoder) have over latent features learned through reconstructing a discrete sample?  It seems that a continuous decoder could enable the learning of discretization-agnostic features.  Is there another motivation for using latent encodings of CNFs for downstream tasks?  The purpose behind their use in enhancing downstream tasks remains somewhat unclear.

The motivation for using local CNF latent encodings could be framed more clearly.

The paper states that a notable limitation for conventional CNF (ln 51): “each field is encoded by a global variable”. However, this statement about cnfs limitations seems to be partially accurate. In fact, this approach for latent space modeling also has some clear advantages. For example, interpolating between two latents to generate novel signals is far more natural with a global latent structure, whereas a local latent structure requires solving the complex problem of finding correspondences between latent points. Thus, the characterization of a tradeoff rather than a limitation may be more appropriate.
Additionally, to address the limitations of a global latent, why not employ an encoder-decoder architecture with gradually decreasing spatial dependency in the latent representation (similar to a UNet)? This approach would provide a final latent that incorporates both local and global information. The rationale for restricting the model to an auto-decoder-style architecture remains unclear.

Overclaiming on Geometry-Appearance Separation in Neural Fields

The paper claims that the proposed method “separates geometry from appearance” in its representation. My understanding is that this refers to the structure of pose-appearance tuples in the latent space. However, how does the method ensure that only appearance information is captured in  $c_i$ ? This seems to rely solely on  $c_i$  being an $SE(n)$ invariant feature. Yet, some relevant geometric features are also invariant (e.g., shape volume), while some equivariant features can relate to appearance (e.g., how an object’s appearance changes are affected by material reflectance features under rotation). Consequently, enforcing a latent structure of invariant and equivariant features may not be sufficient to achieve true separation. Is there empirical evidence to support the above claim about separation?

Unclear reconstruction results

The paper claims: “Results show that ours as well as the baseline models struggle with accurately reconstructing the underlying shape from the SDF point clouds”. Given the inaccuracies in reconstruction, how can the learned features be effectively used for downstream tasks? Additionally, it’s unclear why this model underperforms in reconstruction compared to [3]. Both architectures appear similar (apart from the equivariant features), yet [3] reports more accurate reconstruction results.

Unclear segmentation results

The choice of ShapeNet as the dataset for segmentation evaluation is questionable, as it is an aligned dataset (line 468). A better alternative might be to use non-aligned datasets, such as those used for human-body segmentation in [4] and [5]. Another option would be to unalign ShapeNet by applying a random $SE(3)$ transformation to each data point. Additionally, it’s unclear if the point cloud-specific architectures were also trained with a reconstruction pretext stage. 

Additional comments.

Figure 8 is uninformative on its own without comparison to other methods, showcasing some of the proposed method qualitative benefits/limitations. 

Conditioning with k-nearest neighbors appears to restrict the smoothness of the modeled field to be at most continuous, while the data signals are at least differentiable.

The Steerability property for CNFs has also been defined and utilized in prior works, such as [1] and [2].

[1] Frame Averaging for Equivariant Shape Space Learning. Matan Atzmon, Koki Nagano, Sanja Fidler, Sameh Khamis, Yaron Lipman.

[2] Vector Neurons: A General Framework for SO(3)-Equivariant Networks. Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas Guibas.

[3] Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models.

[4] Approximately Piecewise E(3) Equivariant Point Networks. Matan Atzmon, Jiahui Huang, Francis Williams, Or Litany.

[5] Generalizing neural human fitting to unseen poses with articulated se (3) equivariance. Haiwen Feng, Peter Kulits, Shichen Liu, Michael J Black, and Victoria Fernandez Abrevaya
Grounding Continuous Representations via Equivariant Neural Fields (ENFs) in Geometry by utilising equivalence relations over (latent, coordinate)-pairs
@inproceedings{
anonymous2024grounding,
title={Grounding Continuous Representations in Geometry: Equivariant Neural Fields},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=A4eCzSohhx},
note={under review}
}
Conditional Neural Fields (CNFs) are increasingly being leveraged as continuous signal representations, by associating each data-sample with a latent variable that conditions a shared backbone Neural Field (NeF) to reconstruct the sample. However, existing CNF architectures face limitations when using this latent downstream in tasks requiring fine-grained geometric reasoning, such as classification and segmentation. We posit that this results from lack of explicit modelling of geometric information (e.g. locality in the signal or the orientation of a feature) in the latent space of CNFs. As such, we propose Equivariant Neural Fields (ENFs), a novel CNF architecture which uses a geometry-informed cross-attention to condition the NeF on a geometric variable—a latent point cloud of features—that enables an equivariant decoding from latent to field. We show that this approach induces a steerability property by which both field and latent are grounded in geometry and amenable to transformation laws: if the field transforms, the latent representation transforms accordingly—and vice versa. Crucially, this equivariance relation ensures that the latent is capable of (1) representing geometric patterns faitfhully, allowing for geometric reasoning in latent space, (2) weight-sharing over similar local patterns, allowing for efficient learning of datasets of fields. We validate these main properties in a range of tasks including classification, segmentation, forecasting and reconstruction, showing clear improvement over baselines with a geometry-free latent space.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Geometric Deep Learning Neural Fields Equivariance Representation Learning Latent Point Clouds
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/19188676de2dc21be03ce72a84c78173abb9fd0a.pdf
learning on graphs and other geometries & topologies
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/bd13062c35fad1655f3651b085006d47a27d3715.zip
Grounding Continuous Representations in Geometry: Equivariant Neural Fields
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission