KAN versus MLP on Irregular or Noisy Functions
Kolmogorov-Arnold networks;Multi-layer Perceptrons;KAN;MLP;Irregularization;Noise
Yes
No ethics review needed.
It is possible to include more challenging problems for comparison? It is well established that MLP can model fairly complicated functions.
Experiment codes are provided for reproducibility.

Do provide some insight on what KAN may be good at modeling.
This work empirically compares Kolmogorov-Arnold Networks with Multi-Layer Perceptron on
learning irregular or noisy functions. The experiment results show that KAN do not always perform
the best.
The finding is purely empirical.

The paper does not clearly state the experiment setting in the main text.

The experiment does not provide conclusive results.

The experiment only tries to fit relatively simple functions. The result may not be relevant to real-world problems.
Yes
No ethics review needed.
I do not have a complete understanding of KAN, but I think KAN appears to be a generalization of projection pursuit regression. Is this correct?
- The author compared KAN and MLP on various irregular and noisy functions and experimentally demonstrated in which cases KAN is worse than MLP.
Authors compare the performance of Kolmogorov-Arnold Networks (KAN) and Multi-Layer Perceptron (MLP) networks on irregular or noisy functions. The author experimentally demonstrated that KAN does not always outperform MLP.
- The author merely compared KAN and MLP experimentally but did not analyze why KAN or MLP performs poorly in certain situations.

- The author experimentally demonstrated that KAN is sometimes inferior to MLP. It would be better to propose a new, improved KAN model to address this.

- There are no experiments on high-dimensional functions. In one dimension, both KAN and MLP are likely to approximate well to some extent, but more experiments are needed to explore how they perform in high-dimensional spaces with irregular points.

- If the experiments are conducted only on univariate functions, many models besides MLP can be compared with KAN. It would be beneficial to include other models commonly used in machine learning in the experiments.
Yes
No ethics review needed.
Given that the structures of both MLP and KANs are well-known, a deeper analysis of their capabilities and limitations in the related work section would be beneficial. More thorough research could uncover more significant findings. For instance, some limitations of KANs identified in the paper are not due to the Kolmogorov-Arnold Theorem but rather due to B-spline, a critical component in KANs that is not discussed in the paper at all.
The structure of the paper is clear and well-organized.
The experimental results are clearly presented.
The experiments validate that KANs are not consistently superior to MLP.
The paper conducts a comparative analysis of experiments between MLP and KANs, discussing the outcomes. It challenges the assumption that KANs consistently outperform MLP in modeling mathematical equations, particularly with irregular functions. The experiments involve applying MLP and KAN to various functions—regular, non-differentiable, discontinuous, singular, and coherent oscillation, with and without noise. These functions are single input and single output. Variations include different training sample sizes, iteration counts, and optimizers. The findings demonstrate that KANs do not always surpass MLP.

While this paper serves as a great exploration to KANs and does establish that KANs are not invariably superior to MLP, it falls short by only providing experimental evidence without introducing new theoretical insights or network structures, thus lacking substantial academic contribution.
The experiments could be designed more targeted. For instance, in the experiments for non-differentiability, both functions feature only a single non-differentiable point. A comparison between functions with single versus multiple non-differentiable points would be more insightful, given the focus on the impact of these points.

The discussion lacks depth. Given the simplicity of both the functions and network structures used, there is potential for a more detailed examination of how parameters are trained and the reasons behind specific outcomes.

The discussion section does not yield any intriguing or unexpected conclusions, nor does it propose any novel theories or structures.
Yes
The authors cite a paper with "our work" in the first page, fourth paragraph of Introduction. This seems to be an instance of violation of anonymity of the authors in the double-blind review process.
Yes, Other reasons (please specify below)
If authors can provide some theoretical insights backing the observed empirical results in all or some of the function classes tested, it would make the work more promising and considerable for this conference.
The work has considered different classes of functions with some common irregularities. The empirical comparison is sound, and extensive plots provided let's the reader to compare the performance of tested KAN and MLPs in each case.
In this empirical study, the authors compare the performance of two ad-hoc versions of KAN and MLPs, both with identical parameter counts, in learning ten single-dimensional real functions, with and without added noise.
The comparison primarily centers on the resulting test accuracy curves; however, it lacks the necessary theoretical justification and fundamental analysis to substantiate the findings.

While the authors have structured the text well, the plots are somewhat cluttered and could be presented more effectively for better clarity. 

Overall, the work appears basic and does not demonstrate the level of novelty typically expected from submissions to ICLR.
@inproceedings{
anonymous2024kan,
title={{KAN} versus {MLP} on Irregular or Noisy Functions},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=TH4gKbZS1E},
note={under review}
}
In this paper, we compare the performance of Kolmogorov-Arnold Networks (KAN) and Multi-Layer Perceptron (MLP) networks on irregular or noisy functions. We control the number of parameters and the size of the training samples to ensure a fair comparison. For clarity, we categorize the functions into six types: regular functions, continuous functions with local non-differentiable points, functions with jump discontinuities, functions with singularities, functions with coherent oscillations, and noisy functions. Our experimental results indicate that KAN does not always perform best. For some types of functions, MLP outperforms or performs comparably to KAN. Furthermore, increasing the size of training samples can improve performance to some extent. When noise is added to functions, the irregular features are often obscured by the noise, making it challenging for both MLP and KAN to extract these features effectively. We hope these experiments provide valuable insights for future neural network research and encourage further investigations to overcome these challenges.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Kolmogorov-Arnold networks Multi-layer Perceptrons KAN MLP Irregularization Noise
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/7a69f8434896b72007037e59d4b50b39647ef814.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/30a2db2db7666373b4b40a0cd4e472e02de90058.zip
KAN versus MLP on Irregular or Noisy Functions
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission