Discretized Quadratic Integrate-and-Fire Neuron Model for Direct Training of Spiking Neural Networks
Machine Learning;Neuromorphic Computing;Deep Learning;Computer Vision
Yes
No ethics review needed.
see weakness
1.  A new neuron modeling approach is proposed, along with an appropriate surrogate gradient window.
2.	The effectiveness of this method is demonstrated across multiple datasets.
3.	Evidence is provided that the QIF model can reduce energy consumption and facilitate model convergence.
The author proposes a targeted neuron modeling approach to address existing issues in Spiking Neural Networks (SNNs). This approach reduces the energy consumption of Leaky Integrate-and-Fire (LIF) neurons to some extent and also facilitates faster model convergence. The author conducted validation experiments on both static and dynamic datasets and achieved successful results. This type of Quadratic Integrate-and-Fire (QIF) model could potentially become a widely applied neuron modeling paradigm.
1.	The generalizability of QIF has not been verified. QIF has only been tested on CNN-based models, and it remains unproven whether this model has good generalization to other types of SNNs.
2.	Although visualizations suggest that QIF can guide models to converge quickly, this conclusion is derived post hoc. Is there any theoretical basis for this?
Yes
No ethics review needed.
1.	Could the authors provide a more comprehensive analysis of QIF-specific parameters, such as the resting potential, critical spiking threshold, and decay rate? How do these parameters affect stability and convergence in different scenarios?
2.	Are there any plans to extend the experiments to larger datasets, such as ImageNet, or to more complex network architectures like Spiking ResNet or Transformers? This would significantly strengthen the paper’s claims about the general applicability of QIF neurons.
3.	How does the QIF neuron model compare to LIF in terms of computational and energy efficiency, especially in the context of neuromorphic hardware? Since QIF introduces more complex dynamics, it might face practical limitations when deployed on resource-constrained hardware.
The paper makes a meaningful attempt to address the limitations of the traditional Leaky Integrate-and-Fire (LIF) neuron model by introducing the Quadratic Integrate-and-Fire (QIF) neuron model, which retains more complex nonlinear dynamics that may better approximate biological neurons. This choice is theoretically promising, as it could enhance the modeling capabilities of spiking neural networks (SNNs) by introducing more biologically realistic dynamics. Additionally, the paper introduces a surrogate gradient window tailored to QIF neurons, a practical contribution that improves training stability in spiking neural networks, and presents experimental results on multiple datasets showcasing QIF's potential benefits.
This paper proposes replacing the traditional Leaky Integrate-and-Fire (LIF) neuron model with a Quadratic Integrate-and-Fire (QIF) neuron model in spiking neural networks (SNNs). The authors argue that QIF neurons retain more complex nonlinear dynamics that may better approximate biological neuron behavior. They introduce a custom surrogate gradient window to stabilize the training of QIF neurons and conduct experiments on multiple datasets to demonstrate potential advantages. However, the paper lacks extensive theoretical analysis and validation on larger datasets such as ImageNet and more complex network architectures (more deeper Spiking ResNet and Spiking Transformer). The experimental scope is limited, and there is minimal discussion on hardware implementation challenges, making the current contribution modest.
The paper lacks theoretical depth and sufficient experimental validation for QIF neurons in more complex scenarios. The primary contribution seems limited to replacing LIF neurons with QIF neurons and conducting experiments on small-scale datasets with simpler models. There is no comprehensive analysis of the impact of QIF-specific parameters (such as the resting potential and critical spiking threshold) on network performance or stability. Furthermore, the experiments are limited without testing on larger datasets like ImageNet or deeper, more complex architectures such as Transformers or deeper Spiking ResNet. These limitations make it challenging to assess the broader applicability and stability of QIF neurons in practical scenarios, and the hardware deployment considerations for QIF neurons are not adequately addressed.
Yes
No ethics review needed.
1. Could you provide a more detailed sensitivity analysis of the additional parameters introduced in the QIF model?

2. Could you provide insights into the theoretical foundations that explain the improved energy efficiency?

3. What are the potential solutions for hardware deployment?

4. Please provide all the experiments I mentioned in the Weakness to demonstrate the effectiveness.
1. Significance: The paper addresses the critical issue of energy consumption in neural networks by introducing a novel QIF neuron model. This focus on reducing energy usage is highly relevant and contributes meaningfully to the SNN community.

2. Clarity: The paper is well-written and easy to understand, with clear explanations of the QIF model’s formulation.
The paper introduces a discretized Quadratic Integrate-and-Fire (QIF) neuron model for SNNs to better preserve the complex non-linear dynamics of biological neurons compared to the LIF model. This enhanced model addresses issues like reduced performance and increased energy consumption caused by extraneous spiking activity in LIF-based SNNs.
1. Limited Experimental Validation: The study primarily employs a single, outdated neural network architecture, lacking evaluations on more modern and diverse models such as Transformers. Additionally, experiments do not cover a range of model sizes, which restricts the assessment of the QIF model's generalizability and scalability.

2. Absence of Large-Scale Dataset: The paper does not include evaluations on the ImageNet dataset.

3. Increased Hyperparameter Complexity: Compared to the traditional LIF model, the QIF model introduces additional hyperparameters.  Although parameter insensitivity is claimed, the supporting evidence relies heavily on simple architectures and datasets, which is not convincing.

4. Marginal Performance Improvements: While the QIF model achieves significant energy reductions, its performance gains in terms of accuracy are not consistently superior to existing methods. This raises questions about whether the additional complexity and parameter overhead are justified by the modest performance enhancements.

5. Lack of Neuromorphic Hardware Adaptability Discussion: The paper does not discuss the compatibility and adaptability of the QIF neuron model with neuromorphic hardware.
Yes
No ethics review needed.
1. Line 41 to Line 45. Could you explain what "discretization techniques" really mean? Also, it is confusing that TrueNorth used such a technique in 2015 but the paper said this technique was introduced in 2018. Could you explain in detail how this happens? 

2. Line 207 "Therefore, we introduce our discretized QIF neuron model". The current story is that the "discretization" of the current spiking neuron model is not good so you introduced a new one. However, the proposed neuronal model is still discretized. This makes the motivation of this paper not clear. Could you explain more about the motivation of your method?
1. The methods are validated on two image recognition datasets and several neuromorphic datasets, which make good contributions. Though ImageNet results are not presented, it can be understandable that surrogate gradient learning on such dataset is very expensive.

2. The loss landscape is visualized in the supplementary materials, which is very helpful for better understanding the method
This paper proposed a new spiking neuronal model to overcome the shortcomings of previous discretized neuron model. The method is widely validated on several datasets and compared with other methods.
The context and motivation are very unclear. 

The paper mentions that a technique is used before it is first proposed. See questions for details. The author should explain this.
Yes
No ethics review needed.
The paper could address potential trade-offs between QIF and LIF neuron models, such as computational cost or latency, as these may be factors in deciding the appropriateness of QIF for specific applications.
The introduction of a discretized QIF neuron model addresses a gap in the literature by capturing more complex, non-linear neuronal dynamics, potentially advancing SNNs closer to biological realism. The authors derive an equation to calculate surrogate gradient windows directly from QIF parameters, which minimizes issues with gradient mismatch and naive initialization. This analytical insight is a valuable addition to the SNN field. The model is extensively benchmarked on both static and neuromorphic datasets, demonstrating its generalizability and competitiveness with state-of-the-art methods. The focus on energy efficiency and robustness is particularly relevant for neuromorphic computing. The authors report significant energy efficiency improvements over LIF, along with smoother loss landscapes and faster convergence. This is crucial for the deployment of SNNs in real-world applications where power constraints are critical.
This paper presents the discretized Quadratic Integrate-and-Fire (QIF) neuron model for training deep Spiking Neural Networks (SNNs). The QIF model is proposed as an alternative to the commonly used Leaky Integrate-and-Fire (LIF) neuron model, aiming to retain non-linear, biologically plausible dynamics after discretization. Through extensive evaluation, the paper demonstrates the QIF model's competitive performance on static datasets (CIFAR-10, CIFAR-100) and superior accuracy on several neuromorphic datasets (CIFAR-10 DVS, N-Caltech-101, N-Cars, DVS128-Gesture). Additionally, it shows that the QIF model improves energy efficiency by 20-123% compared to LIF neurons and offers faster convergence due to smoother loss landscapes and greater robustness to hyperparameter selection.
While the paper claims that the QIF model is more biologically plausible, it would benefit from a brief explanation of how the non-linear dynamics in QIF neurons more accurately reflect biological neurons, particularly compared to LIF. The reported 20-123% improvement in energy efficiency is broad, and it is unclear what factors influence this range. Clarifying whether architecture, dataset, or another factor contributes to this variation would improve interpretability. The claim of greater robustness to hyperparameter selection is promising, yet the paper would be strengthened by including more detailed ablation studies to substantiate this claim.
@inproceedings{
anonymous2024discretized,
title={Discretized Quadratic Integrate-and-Fire Neuron Model for Direct Training of Spiking Neural Networks},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=mJ4mgYjDru},
note={under review}
}
Spiking Neural Networks (SNNs) present a promising alternative to traditional Artificial Neural Networks for their potential energy savings. Conventional approaches that use SNNs typically employ a discretized neuron model, which loses the complex, non-linear dynamics found in biological neurons. This can potentially lead to a decrease in model performance and an increase in energy consumption through extraneous spiking activity. To address this issue, we introduce a discretized Quadratic Integrate-and-Fire (QIF) neuron model. The QIF model maintains complex non-linear dynamics, addressing limitations in the widely adopted discretization of the Leaky Integrate-and-Fire (LIF) neurons, which utilize linear dynamics. Our model is evaluated against state-of-the-art approaches on static datasets (CIFAR-10, CIFAR-100) and neuromorphic datasets (CIFAR-10 DVS, N-Caltech-101, N-Cars, DVS128-Gesture), demonstrating competitive performance and improved accuracy. The QIF model achieves a significant energy reduction of up to $123$\% compared to the same model using the LIF neuron model. Additionally, we observe smoother loss landscapes and larger local minima with the QIF model, leading to quicker training convergence. Our findings suggest that the QIF neuron model offers a promising alternative to the widely adopted LIF neuron model.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Machine Learning Neuromorphic Computing Deep Learning Computer Vision
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/6cf75330656dd6ddec7f2dd1c58e231ae4816f9d.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/005f83899c80364e551772b5871ca10a0bb9a380.zip
Discretized Quadratic Integrate-and-Fire Neuron Model for Direct Training of Spiking Neural Networks
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission