Fengbo: a Clifford Neural Operator pipeline for 3D PDEs in Computational Fluid Dynamics
Clifford Algebra;Neural Operator;PDE Modelling;Geometric Machine Learning
Yes
No ethics review needed.
1. Please compare with more recent developed methods
2. Please conducts more experiments on different datasets or PDEs.
The paper presents a novel approach to solving 3D partial differential equations (PDEs) in computational fluid dynamics (CFD) through the use of Clifford Algebra. The use of a pipeline that incorporates only 3D convolutional and Fourier Neural Operator (FNO) layers within a Clifford Algebra framework is a creative combination of mathematical structures and neural network architectures. This uniqueness not only offers a fresh perspective but also has the potential to influence future research in both machine learning and PDE solving. The ability to visualize outputs as physical quantities in 3D space transforms the model into a "whitebox" system. This enhances the interpretability of complex models, which is increasingly important in scientific computing, as it allows for better understanding and trust in the results produced by neural networks.
The paper introduces Fengbo, a novel computational pipeline designed to solve 3D partial differential equations (PDEs) specifically for applications in computational fluid dynamics (CFD). Utilizing Clifford Algebra, Fengbo employs an architecture that consists solely of 3D convolutional and Fourier Neural Operator (FNO) layers, effectively modeling the PDE solution process as a clear mapping from geometric representations to the underlying physics of the problem. Despite its relatively simple architecture with only 42 million trainable parameters, Fengbo demonstrates competitive accuracy, outperforming five out of six models previously proposed in the literature for the same dataset. The architecture achieves this with reduced computational complexity compared to graph-based methods while estimating both pressure and velocity fields. A notable feature of Fengbo is its transparency; the output of each layer can be visualized as objects and physical quantities in 3D space, thereby classifying it as a "whitebox" model. By integrating geometry with physics, Fengbo offers an efficient, interpretable, and physics-aware solution for addressing complex PDEs in CFD applications.
1. Lack of Comprehensive Benchmarking
2. Absence of Generalization and Scalability Discussion
3. Potential Overlook of Limitations
Yes
No ethics review needed.
--> In Table 2, for FNO, the test error is lower than the train error, how is this possible?

--> In Eqs. 20 and 21, the loss function is a combination of the relative $L^2$ norm with the absolute $L^1$ norm. Mathematically, it does not seem reasonable. How do you justify that?

--> I had some concerns, listed in Weakness. Please address them. Thanks.
High quality of writing and figures. Details explanations. A successful extension of Clifford Neural Layers for PDE Modeling to the Navier-Stokes equations for molding fluid dynamics.
This work is essentially the combination of "Clifford Neural Layers for PDE Modeling":

https://arxiv.org/abs/2209.04934

and FNO:

https://arxiv.org/abs/2010.08895

with the extension to computational fluid dynamics (CFD). 3D test cases are considered in this work, with the goal of prediction of the pressure and velocity fields. The complexity of the algorithm, error analysis, and visual comparison between the ground truth and prediction were conducted in this work.
--> The novelty is limited. This is simply just another application of the Clifford Neural Layers for PDE Modeling paper.

--> In the literature review, the classes of PointNet and PointNet++ for deep learning of CFD have been missed. I suggest that the authors take a look and search on Google Scholar to find those articles and perhaps discuss them. Note that PointNet is suitable for unstructured grids and much lighter than graph neural networks since there is no connectivity between nodes.

--> I disagree with the claim of this manuscript saying that their proposed method is appropriate for irregular grids compared to graph neural networks or PointNet because they still convert irregular grids to Cartesian grids and this definitely introduced errors no matter how much you "carefully" convert these data.

--> Following my previous comment, I believe the information listed in Table 4 is misleading. FNO can be used for irregular geometries if one uses geometric transfer. See the following paper:

https://www.jmlr.org/papers/v24/23-0064.html

On the other hand, the proposed method is not inherently designed for irregular geometries, similar to CNNs and FNOs.

--> As a minor comment, it is better to write L2 as the $L^2$ norm
Yes
I have no ethics concerns.
No ethics review needed.
* Line 245: Why is the bivector component left blank in the case of the velocity? The surface information is specially important for the velocity field as it determines the boundary layer dynamics, crucial for drag/lift analysis in aerodynamics. 
* The method's claim of being a white-box model may be overstated. While the final layers may provide some intuitive information during the decoding stage, the true learning of the underlying physics primarily occurs in the 3D Clifford FNO block, which still operates within a latent space. Furthermore, how practical is the interpretability of the final layers beyond offering intuition about the predictions? Have the authors observed any relevant phenomena, such as different energetic modes of the pressure and velocity field across each decoding stage? Without any analysis of this regard, the interpretability claim remains largely qualitative rather than quantitative and has little practical use.
* Table 2: Why MeshGraphNets are not tested for the ShapeNet car dataset? The original paper predicts both the pressure and the momentum field of the fluid.
* Line 388: "[...] being the only architecture reported able to do so while jointly estimating the scalar pressure field and the 3D velocity field". Any of the reported architecture can be trivially modified to include an additional output (the velocity field), so I don’t see this as a specific advantage of the proposed methodology.

Final comment: The presented paper proposes almost the same methodology as GINO ("Geometry-Informed Neural Operator for Large-Scale 3D PDEs" paper) but changing the GNO layers with Clifford algebra layers. From an accuracy and novelty perspective, the results offer only incremental improvements. While the interpretability of the final layers might provide some vague intuition, the paper fails to extract any substantial insights from an engineering or mathematical perspective. The only real novelty of the paper is the reduction in number of parameters, which in my view is insufficient for the standards of this venue. For these reasons, I'd rate this paper as marginally below the acceptance threshold.
* The Clifford algebra architecture is an interesting inductive bias and is generalizable to more complex multidimensional fields. 
* The method has less trainable parameters compared to other techniques.
* The last blocks of the algorithm can be interpreted as velocity and pressure fields, so one can have a visual intuition of how the network learns the final  prediction stage.
The paper presents a new deep learning pipeline to predict the velocity and pressure fields in 3D CFD simulations. The algorithm relies on the use of Clifford algebra layers, a mathematical construction which enables the processing of n-dimensional multivector fields. First, the input fields are upsampled and mixed to translate local to global features. Then, the global information is processed with a FNO-like frequency learning algorithm in a regular voxelized domain. Last, the processed information is decoded to the desired outputs: pressure and velocity fields. This pipeline is tested in the ShapeNet Car and Ahmed Body benchmark datasets and compared with other state-of-the-art architectures.
* The method is limited for steady-state simulations, and only tested for low density/viscosity fluids.
* The performance of the method is very similar in error compared to existing techniques.
* The voxelization of the space might be very inefficient with more complex geometries.
Yes
No ethics review needed.
Please refer to weaknesses section for questions.
* The paper introduces a novel approach by embedding the entire architecture in Clifford Algebra, which allows for a unified treatment of geometric and physical data, enhancing model interpretability and preserving geometric relationships.

* This model provides white-box interpretability by representing intermediate outputs as multivectors, which correlate with physical quantities in 3D space.
The paper introduces Fengbo, a neural operator pipeline that uses Clifford Algebra to solve 3D PDEs in computational fluid dynamics. Fengbo leverages 3D convolutional and Fourier Neural Operator layers within a Clifford Algebra framework to map 3D geometries to physical fields, such as pressure and velocity. It demonstrates competitive accuracy on CFD datasets with fewer parameters and lower computational complexity, while offering interpretability.
* No comparison with the most advanced deep learning-based methods (e.g. transolver, etc.).

* The main results in the paper show better results with fewer parameters, but not the best performance, and it would be better if the performance could be compared with the same parameter Settings. 

* The paper was validated on a limited dataset, and it is hoped that it can be validated on more diverse datasets and tasks (e.g., point cloud, structured mesh, regular grid, etc.), which can be referred to transolver's experimental design.

Minor comments:
* L257"... were the range of the summation of l,m,n isspecified by the kernel size and cin,cout are the ..." has some grammar and typo issues in line 127.

* The definitions of metrics in lines 346 to 363 (formulas (11),(12)) are inconsistent (groud truth and estimated results are used in the denominators respectively).

I would consider revising my rating if the author is able to address my questions and effectively improve on the areas I identified as weaknesses.
Yes
No ethics review needed.
What is the bottleneck that caps the model size at 42M parameters? This does not seem overly large for 3D problems.

Minor, but why are the sizes of the other models in table 2 not listed? How many parameters did they have?

(Very minor recommendation, it's a good idea to give intuition with figures like fig. 2 about the Clifford algebra setup, but figure 3, for example, did not add much information.)
I think the paper has the following strong points:
- it targets non-trivial flow scenarios in three dimensions
- the ShapeNet cars and especially the Ahmet body are interesting use cases
- a nice range of baselines models is compared to
- the underlying theory is complex
This submission targets the learning of 3D flow fields together with pressure distributions by using Clifford algebra. This approach has been proposed in previous work, and the submission at hand extends its implementation, and add geometry and physics blocks that seem to primarily aim for up- and down-sampling.

As the paper is largely extending previous work, it does not include and evaluate simpler cases, but directly focuses on 3D flows. Results are shown for flows around obstacle geometries from ShapeNet cars, and the Ahmet body. 

The results are mixed: in some cases the proposed architecture seems to perform well, but is outperformed by previous work in others. Especially the classic Unet still seems to do a fairly good job, and probably has a much lower computational workload (and simpler implementation).

Overall, I like the direction of the paper: 3D flows are definitely a challenging topic, and important for practical applications. At the same time, the proposed method does not seem overly convincing to me: it is very tailored towards 3D flows, and seems somewhat incremental given the previous work on Clifford based GNNs. With the results in the paper, I would be hesitant to try this approach, and correspondingly, I also find it difficult to really argue for accepting this paper to ICLR. I think with the current, somewhat narrow scope on 3D pressure (+velocity) the submission would be better suited for a more specialized conference or journal.
At the same time, the submission has weak spots:
- the approach seems to be specialized to 3D flows, and I don't see how it would naturally extend to other problems
- the gains in terms of accuracy seem to be mild, which is a pity given the complexity of the approach
- the Clifford algebra comes from previous work, and I have to admit that I don't find it intuitive to work with
- the properties of the baselines are not fully clear (e.g., parameter counts are missing)
Fengbo is a Clifford Algebra pipeline for solving 3D PDEs in CFD. With 42M parameters, it achieves competitive accuracy, outperforms five out of six models and enables 3D visualization, offering an efficient and interpretable geometry-aware solution.
@inproceedings{
anonymous2024fengbo,
title={Fengbo: a Clifford Neural Operator pipeline for 3D {PDE}s in Computational Fluid Dynamics},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=VsxbWTDHjh},
note={under review}
}
We introduce Fengbo, a pipeline entirely in Clifford Algebra to solve 3D partial differential equations (PDEs) specifically for computational fluid dynamics (CFD). Fengbo is an architecture composed of only 3D convolutional and Fourier Neural Operator (FNO) layers, all working in 3D Clifford Algebra. It models the PDE solution problem as an interpretable mapping from the geometry to the physics of the problem. Despite having just few layers, Fengbo achieves competitive accuracy, superior to 5 out of 6 proposed models reported in \cite{li2024geometry} for the $\emph{ShapeNet Car}$ dataset, and it does so with only 42 million trainable parameters, at a reduced computational complexity compared to graph-based methods, and estimating jointly pressure \emph{and} velocity fields. In addition, the output of each layer in Fengbo can be clearly visualised as objects and physical quantities in 3D space, making it a whitebox model.  
By leveraging Clifford Algebra and establishing a direct mapping from the geometry to the physics of the PDEs, Fengbo provides an efficient, geometry- and physics-aware approach to solving complex PDEs.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Clifford Algebra Neural Operator PDE Modelling Geometric Machine Learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/e71185c163c544a003c096547439e38434eee27e.pdf
applications to physical sciences (physics, chemistry, biology, etc.)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Fengbo: a Clifford Neural Operator pipeline for 3D PDEs in Computational Fluid Dynamics
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission