SpikSSD: Better Extraction and Fusion for Object Detection with Spiking Neuron Networks
Spiking Neural Networks;Object Detection;Membrane-based Deformed Shortcut;Spiking Bi-direction Fusion Module
Yes
No ethics review needed.
1.There could be experiments on MS-COCO dataset and real-world images for validation qualitatively.

2. Could explain the following part with efficient manner? "SBFM has a PAN-like structure but with 5 feature maps. There could be a more efficient way of feature fusion other than the proposed SBFM."
1.SpikSSD shows improvements in energy efficiency, achieving a 10% firing rate with competitive accuracy, which is beneficial for power-constrained applications.
2.Sufficient mathematical proof of the network structure mathematically, which proves the liability of the proposed MDS-ResNet.
3.Experiments on GEN1 and VOC proves the performance of SpikeSSD. On both dataset SpikeSSD proves to be the best SNN-based object detection network.
The paper presents SpikSSD, an SNN-based object detection network. SpikSSD introduces a spiking residual network (MDS-ResNet) for feature extraction and Spiking Bi-direction Fusion Module (SBFM) for multi-scale feature fusion. Experiments on GEN1 and VOC dataset indicates the best performance of SNN-based object detection.
1.For object detection on images, the authors only provide experiment results on VOC dataset, which is insufficient since MS-COCO dataset is the mainstream dataset for this task. VOC dataset has much fewer images, which lacks of utility. There should be experiments on MS-COCO dataset and real-world images for validation qualitatively.

2.SBFM is too complicated for SNN-based network. SBFM fuses different scales of feature maps, but it is too complicated for training. To meet the requirement of SSD detection head, multiple features need to be generated. SBFM has a PAN-like structure but with 5 feature maps. There could be a more efficient way of feature fusion other than the proposed SBFM.

3.The MDS-ResNet primarily extends SNN modules on mature ANN networks, which lacks new design or new insight on SNN. Core ideas like membrane-based shortcuts and spiking feature fusion have seen prior application.

SpikSSD provides a model that is interesting but the contributions are incremental, offering refinements on existing modules rather than making novel breakthroughs. Additionally, without real-world experiments, it is hard to identify the claimed efficiency gains. The paper would benefit from hardware validation and further simplification for real-world deployment or application, rather than merely experimenting on certain dataset.
Yes
No ethics review needed.
1. Can you make a more detailed comparison of the EMS and MDS architectures in terms of formulas? And show why the EMS architecture cannot be fused to the spike operation?
2. Regarding the calculation of energy consumption, can you give a discussion on how to deploy it on real hardware? Similarly, I would like to see a comparison between MDS and other model architectures?
3. What do you think of the method of gradient identity analysis? Are there any problems with this method's series of assumptions in discrete models such as SNN?
1. The author's analysis of the spike operation in the algorithm is very accurate, and based on this, he designed a spike-friendly MDS architecture.
2. The method proposed by the authors has good performance and energy efficient. (more elegant than EMS method~)
3. The author provides a relatively complete theoretical analysis.
This paper presents an MDS architecture for solving the problem of SNN target detection. This approach enables the model to further utilize spike operations to reduce computing power. The authors validated it on multiple target detection datasets.
1. This article appears to be incremental work.
2. See Question part.
Yes
No ethics review needed.
Please see the weaknesses part.
1. The proposed MDS-ResNet adjust uses tdBN in the shortcut to adjust the distribution of input membrane synaptic to alleviate the problems brought by gradient vanishing and explosion.
2. This paper gives a theoretical analysis of gradient vanishing/explosion for MDS-ResNet.
This paper proposes an SNN-based single-shot multi-box detector SpikSSD. In the SpikSSD, this paper proposes a backbone MDS-ResNet. The MDS-ResNet adjusts the distribution of the input membrane synaptic to avoid gradient vanishing and explosion. Besides, this paper also proposes a Spiking Bi-direction Fusion Module (SBFM) to fuse spiking features in a bi-directional way. The proposed method is tested on GEN1 and VOC 2007 datasets, where the SpikSSD achieves good performance with only around 10% firing rate.
1. This paper claims they propose a backbone MDS-ResNet based on spiking neural networks. Is the backbone specifically designed for object detection? If so, what part of MDS-ResNet is specifically designed for objection detection? If not, it should be evaluated on classification datasets such as ImageNet [1] to show the generalization of the proposed backbone [2-5].
2. In the area of object detection, COCO [6] is a general dataset, why not evaluate the proposed method on this dataset?
3. What's the motivation for the proposed bi-directional fusion? What challenges are handled by the bi-directional fusion? It would be better to introduce more about that.
4. The experimental table of the paper shows the proposed method has a low energy. How is the low energy consumption realized? On general computational devices or specifically designed devices? If the low energy depends on specifically designed devices, how is the low energy realized?

[1] J. Deng, et al., Imagenet: A large-scale hierarchical image database. CVPR 2009.  
[2] Y. Li, et al., Differentiable spike: Rethinking gradient-descent for training spiking neural networks. NeurIPS 2021.  
[3] W. Fang, et al., Deep residual learning in spiking neural networks. NeurIPS 2021.  
[3] H. Zheng, et al., Going deeper with directly-trained larger spiking neural networks. AAAI 2021.  
[4] X. Yao, et al., GLIF: A unified gated leaky integrate-and-fire neuron for spiking neural networks. NeurIPS 2022.  
[5]  W. Fang, et al., Parallel spiking neurons with high efficiency and ability to learn long-term dependencies. NeurIPS 2023.  
[6] T. Lin, et al., Microsoft COCO: common objects in context. ECCV 2014.
Yes
No ethics review needed.
Certain sections lack clarity, particularly regarding the mechanisms of MDS and SBFM. Visualizations of neuron firing patterns and more illustrative figures could help elucidate these mechanisms. Additionally, providing more detail in the methodology could allow for better reproducibility.
The paper highlights the low energy consumption of SpikSSD, reporting approximately 10% firing rates, which is notable in comparison to conventional ANNs.
This paper introduces SpikSSD, a Spiking Single Shot Multibox Detector that leverages a fully spiking backbone network (MDS-ResNet) and a novel Spiking Bi-direction Fusion Module (SBFM) for object detection in Spiking Neural Networks (SNNs). The model shows strong performance on the GEN1 and PASCAL VOC 2007 datasets, achieving a mean average precision (mAP) comparable to or better than existing SNN-based models with a low firing rate.
The results are primarily limited to GEN1 and VOC 2007, which may not fully represent the performance across diverse object detection tasks. The paper could be strengthened by including more diverse datasets or a comprehensive analysis of SpikSSD's generalization capabilities. While the paper introduces the MDS to stabilize membrane synaptic inputs, the theoretical foundations are only briefly discussed. A more rigorous exploration of why MDS effectively stabilizes membrane potential and prevents gradient vanishing/explosion would strengthen the paper’s contributions. Additionally, it’s unclear how the MDS compares with existing methods for managing synaptic input variance in SNNs. Although the results are promising, the comparisons are limited to SNN-based models. The paper would benefit from a broader comparison with non-SNN state-of-the-art object detectors on similar datasets to better highlight SpikSSD's competitiveness in real-world applications. Given that SNNs are often measured against ANN counterparts, direct ANN comparisons could showcase where SNNs might still be falling short. While SBFM achieves bi-directional fusion, the paper lacks a detailed justification of why this method is uniquely advantageous in SNNs as opposed to standard fusion techniques. It would also be helpful to clarify the specific benefits of using a spiking-only architecture over incorporating mixed approaches if they offer potential improvements in performance. The paper does not include ablation studies to validate the specific contributions of MDS-ResNet and SBFM. Without this, it is difficult to assess whether the performance gains are directly attributable to these components. Such studies could better clarify the impact of each proposed module.
Yes
No ethics review needed.
Please refer to the weaknesses.
1. This paper analyzes in detail the gradient vanishing or explosion for MDS-ResNet.
2. The proposed SpikSSD model achieves state-of-the-art mAP metric performance on GEN1 and VOC2007 datasets compared to existing spiking object detection methods.
This paper proposes a spiking neural network backbone MDS-ResNet, and a spiking object detection model SpikSSD. Based on EMS-ResNet, the proposed MDS-ResNet modifies the shortcut connection in the residual blocks. The SpikSSD combines the proposed bi-directional feature fusion and MDS-ResNet. Experimental results show that the proposed SpikSSD outperforms existing spiking object detection methods.
1. In the method section, the authors spend a lot of words introducing the proposed backbone MDS-ResNet. However, MDS-Blocks 1, 2, and 4 are already utilized in MS-ResNet and EMS-ResNet, while only MDS-Block 3 is newly proposed, and it closely resembles MDS-Block 2. Moreover, the gradient analysis of MDS-ResNet is similar to EMS-ResNet. Therefore, I believe that the MDS-ResNet architecture should not be regarded as a main contribution of this paper. I suggest that the authors detail why MDS-Block is more effective for object detection tasks than MS-Block and EMS-Block.
2. The SpikSSD is only evaluated on VOC 2007 and GEN1 datasets, while EMS-ResNet is evaluated mainly on COCO 2017 dataset. I wonder if SpikSSD also works well on COCO 2017. If possible, please provide the experimental results on COCO 2017 dataset.
3. Figure 1 illustrates the firing patterns of MDS-ResNet and EMS-ResNet. However, it does not indicate on which task or dataset the evaluation is performed. Furthermore, it is not clear whether this phenomenon holds for all tasks or only for specific tasks.
@inproceedings{
anonymous2024spikssd,
title={Spik{SSD}: Better Extraction and Fusion for Object Detection with Spiking Neuron Networks},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=hpZ5zpudH8},
note={under review}
}
As the third generation of neural networks, Spiking Neural Networks (SNNs) have gained widespread attention due to their low energy consumption and biological interpretability. Recently, SNNs have made considerable advancements in computer vision. However, efficiently conducting feature extraction and fusion under the spiking characteristics of SNNs for object detection remains a pressing challenge. To address this problem, we propose the SpikSSD, a novel Spiking Single Shot Multibox Detector. Specifically, we design a full-spiking backbone network, MDS-ResNet, which effectively adjusts the membrane synaptic input distribution at each layer, achieving better spiking feature extraction. Additionally, for spiking feature fusion, we introduce the Spiking Bi-direction Fusion Module (SBFM), which for the first time realizes bi-direction fusion of spiking features, enhancing the multi-scale detection capability of the model. Experimental results show that SpikSSD achieves 40.8\% mAP on the GEN1 dataset and 76.0\% mAP@0.5 on the VOC 2007 dataset with only around 10\% firing rate, outperforming existing SNN-based approaches at ultralow energy consumption. This work sets a new benchmark for future research in SNN-based object detection. Our code is publicly available in supplementary materials.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Spiking Neural Networks Object Detection Membrane-based Deformed Shortcut Spiking Bi-direction Fusion Module
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/42b69e52e972fe18eb2142a4b72749e63e54c191.pdf
applications to neuroscience & cognitive science
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/34d55b5b4814deb37a82bcbe1f3d97339d0dd397.zip
SpikSSD: Better Extraction and Fusion for Object Detection with Spiking Neuron Networks
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission