Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks
Malu Zhang Jiadong Wang Burin Amornpaisannon Zhixuan Zhang VPK Miriyala Ammar Belatreche Hong Qu Jibin Wu Yansong Chua Trevor E. Carlson Haizhou Li
cs.NE cs.LG



Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks
    Malu Zhang,
        Jiadong Wang,
        Burin Amornpaisannon,
        Zhixuan Zhang,
        VPK Miriyala,
        Ammar Belatreche,
        Hong Qu,
        Jibin Wu,
        Yansong Chua,
        Trevor E. Carlson, Senior Member, IEEE,
        and Haizhou Li, Fellow, IEEE
This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may
no longer be accessible.

This research is supported by Programmatic Grant No. A1687b0033 from the Singapore Government's Research, Innovation and Enterprise 2020 plan (Advanced Manufacturing and Engineering domain), the National Key Research and Development Program of China (2018AAA0100202), the Zhejiang Lab (Grant No. 2019KC0AB02).

M. Zhang, J. Wang, J. Wu, Y. Chua and H. Li are with the Department of Electrical and Computer Engineering, National University of Singapore, Singapore.
B. Amornpaisannon, V. P. K. Miriyala, and T. Carlson are with the School of Computing, Department of Computer Science, National University of Singapore, Singapore.
Z. Zhang, and H. Qu are with the School of Computer
	Science and Engineering, University of Electronic Science and Technology
	of China, Chengdu 610054, China.
A. Belatreche is with the Department of Computer and Information
	Sciences, Faculty of Engineering and Environment, Northumbria
	University, Newcastle upon Tyne NE1 8ST, U.K.

M. Zhang, J. Wang, B. Amornpaisannon and Z. Zhang contributed equally to this work.
    December 30, 2023
=======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



Spiking Neural Networks (SNNs) use spatio-temporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. Motivated by the success of deep learning, the study of Deep Spiking Neural Networks (DeepSNNs) provides  promising directions for artificial intelligence applications. However,  training of DeepSNNs is not straightforward because the well-studied error back-propagation (BP) algorithm is not directly applicable. In this paper, we first establish an understanding as to why error back-propagation does not work well in DeepSNNs. To address this problem, we propose a simple yet efficient Rectified Linear Postsynaptic Potential function (ReL-PSP) for spiking neurons and propose a Spike-Timing-Dependent Back-Propagation (STDBP) learning algorithm for DeepSNNs. In STDBP algorithm, the timing of individual spikes is used to convey information (temporal coding), and learning (back-propagation) is performed based on spike timing in an event-driven manner. Our experimental results show that the proposed learning algorithm achieves state-of-the-art classification accuracy in single spike time based learning algorithms of DeepSNNs. Furthermore, by utilizing the trained model parameters obtained from the proposed STDBP learning algorithm, we demonstrate the ultra-low-power inference operations on a recently proposed neuromorphic inference accelerator. Experimental results show that the neuromorphic hardware consumes 0.751 mW of the total power consumption and achieves a low latency of 47.71 ms to classify an image from the MNIST dataset. Overall, this work investigates the contribution of spike timing dynamics  to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.




Spiking neural networks, Deep neural networks, Spike-timing-dependent learning, Event-driven, Neuromorphic hardware




§ INTRODUCTION

The success of Deep Neural Networks (DNNs) is attributed to their underlying deep hierarchical structure that learns the  representations of big data with multiple levels of abstraction <cit.>. With DNNs, we have advanced the state-of-the-art in many machine learning tasks by leaps and bounds, such as image recognition<cit.>, speech recognition<cit.>, natural language processing <cit.>, and medical diagnosis <cit.>. However, training of DNNs generally requires high computing resources (e.g., GPUs and computing clusters). Therefore, in power-critical computing platforms, such as edge computing, the implementation of DNNs is drastically limited <cit.>. Spiking Neural Networks (SNNs) provide a low-power alternative to neural network implementation. It is designed to emulate brain computing, that has the potential to provide computing capabilities equivalent to that of DNNs on an ultra-low-power spike-driven neuromorphic hardware <cit.>. However, due to the relatively shallow network structures, SNNs have yet to match the performance of their DNN counterparts in pattern classification tasks on standard benchmarks<cit.>. 
[This research is supported by Programmatic Grant No. A1687b0033 from the Singapore Government's Research, Innovation and Enterprise 2020 plan (Advanced Manufacturing and Engineering domain), the National Natural Science Foundation of China (Grant No. 61976043 and 61573081), the National Key Research and Development Program of China (2018AAA0100202), the  Zhejiang Lab (Grant No. 2019KC0AB02).
 
 Email: maluzhang@nus.edu.sg ]

There has been a growing interest in the implementation of deep structures for SNNs (DeepSNNs)
<cit.>. Unfortunately, the training of DeepSNNs is not straightforward as the well-studied error back-propagation (BP) learning algorithm is not directly applicable due to the complex temporal dynamics and the non-differentiable spike function. Addressing the issue of DeepSNN training,  successful implementations can be grouped into three categories.

The first category is ANN-to-SNN conversion methods, where we first train an ANN, then approximate the pre-trained ANN with an SNN equivalent <cit.>. The idea of such ANN-to-SNN approximation  benefits from the state-of-the-art ANN training algorithms and uses an equivalent SNN  to approximate the performance of the pre-trained ANN. However, the conversion suffers from loss of accuracy due to the approximation. Despite much progress, such as weight/activation normalization <cit.>, and adding noise to the model <cit.>, the solutions are far from being perfect. Furthermore, as the spiking rate of a neuron is typically used to encode analog activity of a classical artificial neuron, a high level of activation requires a high spike rate, which then hides the timing information of the spiking activity, and is energy-intensive <cit.>. Furthermore,  the inference time of the spike rate based encoding scheme is another problem <cit.>.

The second category is the membrane potential driven learning algorithms, treating the neuron's membrane potential as differentiable signals which are used to solve the non-differentiable problems of spikes with using surrogate derivatives <cit.>. For example, <cit.> back-propagate errors based on the membrane potential at a single time step, which ignores the temporal dependency, but only use signal values at the current time instance. To address this problem, SLAYER <cit.> and STBP <cit.> train DeepSNNs with surrogate derivatives based on the idea of Back-propagation Through Time (BPTT) algorithm. While competitive accuracies are reported on the MNIST and CIFAR10 datasets <cit.>, the computational and memory requirements of these algorithms are high for BPTT because the entire sequence must be stored to compute the gradients accurately.

The third category is the spike-driven learning algorithms, which uses the timing of spikes as the relevant signals for controlling synaptic changes. The typical examples include SpikeProp <cit.> and its derivatives <cit.>. These methods apply a linear assumption that the neuron's membrane potential increases linearly in the infinitesimal time around the spike time, allowing the calculation of the derivative of the spike function and facilitating the implementation of back-propagation in multi-layer SNNs. Recently, Mostafa <cit.> applied non-leaky integrate-and-fire neurons to avoid the problem of the non-differentiable spike function, and the work showed competitive performance on MNIST dataset. The performance of SNN with spike-driven learning algorithm is further improved by <cit.> and <cit.>. However, the existing spike-driven learning algorithms suffer from certain limitations, such as the problems of dead neurons and gradient exploding <cit.>. For example, in <cit.>, some constraints are imposed on the synaptic weights to overcome the dead neuron problem, and a gradient normalization strategy is used to overcome the problem of gradient exploding. These complex training strategies limit the scalability of these learning algorithms. 

Among the existing learning algorithms, the spike-driven learning algorithms perform the SNNs training in a strictly event-driven manner, and are compatible with the temporal coding in which the information is carried by the timing of individual spikes in a very sparse manner; Hence, spike-driven learning algorithms hold the potential of enabling training and inference in low-power devices. Recently, several neuromorphic architectures have been proposed for accelerating the SNN inference and training efficiently <cit.>. Currently, the most prominent among them is Intel's Loihi <cit.>, which can support on-chip learning with a wide range of spike-timing-dependent-plasticity (STDP) rules. During SNN acceleration, Loihi is claimed to be 1000× faster than the general-purpose processors such as CPUs and GPUs, while using much less power. In addition, Srivatsa et al, <cit.> recently proposed a neuromorphic inference accelerator called YOSO. YOSO can significantly leverage the sparse spiking activity in SNNs and facilitates the ultra-low-power inference operations with high classification accuracies <cit.>. With these considerations, we develop an effective spike-driven learning algorithms for DeepSNNs with the  following contributions in this paper:

1) We thoroughly analyze the issues that make the well-studied BP algorithm incompatible for training SNNs, including the problems of non-differentiable spike generation function, gradient exploding and dead neuron. Building on such an understanding, we put forward a Rectified Linear Postsynaptic Potential function (ReL-PSP) for spiking neurons to resolve these problems. 

2) Based on the proposed ReL-PSP, we derive a new spike-timing-dependent BP algorithm (STDBP) for DeepSNNs. In this algorithm, the timing of spikes is used as the information-carrying quantities, and learning happens only at the spike times in a totally event-driven manner.

3) As STDBP with ReL-PSP is highly scalable, we extend it to convolutional spiking neural network (C-SNN), and achieve an accuracy of 99.4% on the MNIST dataset. To the best of our knowledge, this is the first implementation of a C-SNN structure in the area of single-spike-timing-based supervised learning algorithms, with a state-of-the-art accuracy performance. 

4) Lastly, using the weights obtained through the proposed STDBP learning algorithm, we demonstrate the ultra-low-power inference operations by accelerating our SNN model on the neuromorphic inference accelerator, YOSO. 

Overall, this work not only provides a novel perspective to investigate the significance of spike timing dynamics in information coding, synaptic plasticity, and decision making in SNNs-based computing paradigm, but also demonstrates the possibility of realizing low power inference operations with high classification accuracy.



§ PROBLEM ANALYSIS


Error back-propagation (specifically stochastic gradient descent) is the workhorse for the remarkable success of DNNs. However, as shown in Fig. <ref>, the dynamics of a typical artificial neuron in DNNs and that in SNNs is rather different, therefore the well-studied BP algorithm cannot be directly applied to DeepSNNs due to non-differentiable spike function, exploding gradients, and dead neurons. In the following, we will discuss these issues in depth.


Consider a fully connected DeepSNN. For simplicity, each neuron is assumed to emit at most one spike. In general, the membrane potential V_j^l of neuron j in layer l with N presynaptic connections can be expressed as,
 
 
    V^l_j(t)=∑_i^Nω_ij^lε(t-t_i^l-1)-η(t-t^l_j)

where t_i^l-1 is the spike of the ith neuron in layer l-1, and ω^l_ij is the synaptic weight of the connection from neuron i (in l-1 layer) to neuron j (in l layer). 
Each incoming spike from neuron i will induce a postsynaptic potential (PSP) at neuron j, and the kernel ε(t-t_i^l-1) is used to describe the PSP generated by the spike t_i^l-1. Hence each input spike makes a contribution to the membrane potential of the neuron as described by ω_ij^lε(t-t_i^l-1) in Eq. <ref>. There are several PSP functions, and a commonly used one is  alpha function which is defined as

    ε(t)=t/τexp(1-t/τ)              t>0


Fig. <ref>(a) shows the alpha-PSP response function. As shown in Fig. <ref>(b), integrating the weighted PSPs gives the dynamics of the membrane potential V_j^l(t). The neuron j will emit a spike when its membrane potential V_j^l(t) reaches the firing threshold ϑ, as mathematically defined in the spike generation function ℱ:

    t_j^l = ℱ{ t |V_j^l(t)=ϑ, t ≥ 0 }

 Once a spike is emitted, the refractory kernel η(t-t^l_j) is used to reset the membrane potential to resting. 



To train SNNs using BP, we need to compute the derivative of the postsynaptic spike time t^l_j with respect to a presynaptic spike time t^l-1_i and synaptic weight ω^l_ij of the corresponding connection:


    ∂ t_j^l/∂ω_ij^l=∂ t_j^l/∂ V_j^l(t_j^l)∂ V_j^l(t_j^l)/∂ω_ij^l    if      t_j^l>t_i^l-1



    ∂ t_j^l/∂ t_i^l-1=∂ t_j^l/∂ V_j^l(t_j^l)∂ V_j^l(t_j^l)/∂ t_i^l-1    if      t_j^l>t_i^l-1


 Due to the discrete nature of the spike generation function (Eq. <ref>), the difficulty of Eq. <ref> lies in solving the partial derivative ∂ t_j^l/∂ V_j^l(t_j^l) , which we refer to as the problem of non-differentiable spike function. Existing spike-driven learning algorithms <cit.> assume that the membrane potential V^l_j(t) increases linearly in the infinitesimal time interval before spike time t_j. Then, ∂ t_j/V_j(t) can be expressed as 

    ∂ t_j^l/∂ V_j^l(t_j^l)=-1/∂ V_j^l(t_j^l)/∂ t_j^l=-1/∑_i^Nω^l_ij∂ε(t_j^l-t_i^l-1)/∂ t_j^l

with 

    ∂ε(t_j^l-t_i^l-1)/∂ t_j^l=exp(1-(t_j^l-t_i^l-1)/τ)/τ^2(τ+t_i^l-1-t_j^l)


The exploding gradient problem occurs when ∂ V_j^l(t_j^l)/∂ t_j^l≈0 i.e. the membrane potential is reaching the firing threshold, emitting a spike (Fig. <ref>b). Since ∂ V_j(t_j)/∂ t_j is the denominator in Eq. <ref>, this causes Eq. <ref> to explode with large weight updates. Despite the progress, such as adaptive learning rate <cit.> and dynamic firing threshold <cit.>, the problem has not been fully resolved. 

From Eq. <ref> and Eq. <ref>, when the presynaptic neuron does not emit a spike, the error cannot be back propagated through ∂ V_j^l(t^l_j)/∂ t_i^l-1. This is the dead neuron problem. This problem also exists with analog neurons with ReLU activation function in DNNs. However, due to the leaky nature of the PSP kernel and spike generate mechanism, spiking neurons encounter a more serious dead neuron problem. As shown in Fig. <ref>(c), there are three input spikes, and the neuron emits a spike with large synaptic weights (blue). With slightly reduced synaptic weights, the membrane potential stays sub-threshold and the neuron becomes a dead neuron (green). When the neuron does not spike, no errors can back-propagate through it. The problem of dead neuron is fatal in spike-driven learning algorithms.



§ SPIKING NEURON MODEL AND LEARNING ALGORITHM


In this section, we describe how the above challenges may be overcome, thereby a DeepSNN may still be trained using BP. To this end, we introduce Rectified Linear Postsynaptic Potential function (ReL-PSP) as a new spiking neuron model, and present a spike-timing-dependent back propagation (STDBP) learning algorithm that is based on ReL-PSP.



 §.§ ReL-PSP based spiking neuron model



As presented in Section <ref>, BP cannot be directly applied in DeepSNNs due to problems of non-differentable spike function, exploding gradient and dead neuron. To overcome the problems, we propose a simple yet efficient Rectified Linear Postsynaptic Potential (ReL-PSP) based spiking neuron model,  whose dynamics is defined as follows,

    V^l_j(t)=∑_i^Nω_ij^lK(t-t_i^l-1)

whereby K(t-t_i^l-1) is the kernel of the PSP function, which is defined as 

    K(t-t_i^l-1) =
    
    t-t_i^l-1  if   t>t_i^l-1
     
    0         otherwise

As shown in Fig. <ref>(a), given an input spike at t_i^l-1, the membrane potential after t_i^l-1 is a linear function of time t. Since the shape of the proposed PSP function resembles that of a rectified linear function, we name it the ReL-PSP function. In the following, we will analyze how the proposed neuron model solves the above-mentioned problems.




  §.§.§ Non-differentiable spike function

As shown in Fig. <ref>b, due to the linearity of the ReL-PSP, the membrane potential V_j^l(t) increases linearly prior to spike time t_j^l. The linearity is a much desired property from a postsynaptic potential function. We can now directly use Eq. <ref> to compute ∂ t_j^l/∂ V_j^l(t_j^l). This resolves the issue of non-differentiable spike generation. 


    ∂ t_j^l/∂ V_j^l(t_j^l)   =-1/∂ V_j^l(t_j^l)/∂ t_j^l
       = -1/∑_i^N ω^l_ij∂ K(t_j^l-t_i^l-1)/∂ t_j^l
       =-1/∑_i^N ω^l_ij    if    t_j^l>t_i^l-1


The precise gradients in BP provide the necessary information for network optimization, which is the key to the performance of DNNs. Without having to assume linearity, we use the precise value of ∂ t_j^l/∂ V_j^l(t_j^l) instead of approximating it, and avoid accumulating errors across multiple layers.



  §.§.§ Gradient explosion

Exploding gradient occurs when the denominator in Eq. <ref> approaches 0. In this case, the membrane potential just reaches the firing threshold at spike time, and is caused by the combined effect of ω_ij^l and partial derivative of the PSP function. Alpha-PSP function has zero gradient at its peak. It is clear that  ReL-PSP function has a lower chance of zero gradient than alpha-PSP function. As the partial derivative of ReL-PSP, ∂ K(t_j^l-t_i^l-1)/∂ t_j^l, is always equal to 1, Eq. <ref> can be expressed as -1/∑_i^N ω^l_ij. 


As ∑_i^N ω^l_ij may still be close to 0, the exploding gradient problem may not be completely solved. However, from Eqs. <ref> and <ref>, we obtain the spike time t_j^l as a function of input spikes and synaptic weights ω^l_ij, and the spike time t_j^l can be calculated as,

    t_j^l=ϑ+∑_i^Nω_ij^lt_i^l-1/∑_i^Nω_ij^l


Should the ∑_i^Nω_ij^l be close to 0, the spike t_j^l will be emitted late, and may not contribute to the spike  t_j^l+1 in the next layer. Therefore, the neuron j in the l layer does not participate in error BP, and does not result in exploding gradient. 



  §.§.§ Dead neuron

In neural networks, sparse representation (few activated neurons) has many advantages, such as information disentangling, efficient variable-size representation, linear separability etc. However, sparsity may also adversely affect the predictive performance. Given the same number of neurons, sparsity reduces effective capacity of the model <cit.>.
Unfortunately, as shown in Fig. <ref>(c), due to the leaky nature of the alpha-PSP and the spike generation mechanism, such a spiking neuron is more likely to suffer from the dead neuron problem. 

As shown in Fig. <ref>(c), with the ReL-PSP kernel, the PSP increases over time within the simulation window T_max. Hence the neuron with a more positive sum of weights fires earlier than one with a less positive sum, with lower probability of becoming a dead neuron. Overall, the proposed ReL-PSP greatly alleviates the dead neuron problem as the PSP does not decay over time, while maintaining a sparse representation to the same extent of the ReLU activation function.



 §.§ Error backpropagation


Given a classification task with n categories, each neuron in the output layer is assigned to a category. When a training sample is presented to the neural network, the corresponding output neuron should fire the earliest. Several loss functions can be constructed to achieve this goal <cit.>. In this work, the cross-entropy loss function is used. To minimise the spike time of the target neuron, at the same time,   maximise the spike time of non-target neurons, we use the softmax function on the negative values of the spike times in the output layer: p_j=exp(-t_j)/∑_i^n exp(-t_i). The loss function is given by,  

    L(g,t^o)=-lnexp(-t^o[g]))/∑_i^nexp(-t^o[i])

where t^o is the vector of the spike times in the output layer and g is the target class index<cit.>.

The loss function is minimised by updating the synaptic weights across the network. This has the effect of delaying or advancing spike times across the network. The derivatives of the first spike time t_j^l with respect to synaptic weights ω_ij^l and input spike times t_i^l-1 are given by 


    ∂ t_j^l/∂ω_ij^l=∂ t_j^l/∂ V_j^l(t_j^l)∂ V_j^l(t_j^l)/∂ω_ij^l=  t_i^l-1-t_j^l/∑_i^N ω^l_ij  if    t_j^l>t_i^l-1



    ∂ t_j^l/∂ t_i^l-1=∂ t_j^l/∂ V_j^l(t_j^l)∂ V_j^l(t_j^l)/∂ t_i^l-1=ω^l_ij/∑_i^N ω^l_ij  if   t_j^l>t_i^l-1


Following Eq. <ref> and Eq. <ref>, a standard BP can be applied  for DeepSNNs training.





§ HARDWARE MODEL



To put the proposed spiking neuron model and STDBP learning algorithm into action, we evaluate them on a hardware architecture for pattern classification tasks. We first implement them in spiking neural networks for inference operations on YOSO platform <cit.>. YOSO is specifically designed for accelerating temporal coding based SNN models with sparse spiking activity. It was shown that YOSO facilitates ultra-low-power inference operations (< 1 mW) as a neuromorphic accelerator with state-of-the-art performance.

In this section, we will describe the hardware architecture of YOSO, the technique that maps SNNs on YOSO, and the simulation methodology that evaluates the YOSO performance during inference operations.



 §.§ Hardware Architecture



As shown in Fig. <ref> (a), YOSO is a Network-On-Chip (NoC) architecture with multiple processing elements (PEs) connected in a mesh topology. Each PE has a router, which facilitates the propagation of spikes from one PE to another. As shown in Fig. <ref> (b), each PE in the NoC consists of four static random-access memories (SRAMs), a memory interface, a core, a router interface, first-in-first-out (FIFO) buffers to facilitate communication between router and router interface. The router sends or receives the spikes to or from other PEs in the NoC. During the initialization phase, i.e. when the accelerator is downloading the model to be run, the router sends the spikes to SRAMs directly via router and memory interfaces. During the inference phase, the router sends the spikes to the core, where all the computations take place. Note that YOSO <cit.> can only accelerate inference operations and training needs to be performed offline. 

The SRAMs are used to store all the information required for processing the incoming spikes and for generating the output spikes. As shown in Fig. <ref> (b), the SRAMs can communicate with the core and router interface via a memory interface. The accumulated weight, neuron, weight, and spike address SRAMs store accumulated weights, neuron potentials, weights between two layers, and the spike addresses of the neurons allocated for that PE, respectively. 

As shown in Fig. <ref> (c), the core consists of three modules–load, compute, and store.  The load module is responsible for decoding the information encoded in incoming spikes. A spike processing algorithm introduced in <cit.> is used to encode the information in spike packets that can read by the load module. After decoding the information in incoming spikes, the load module generates read requests to different SRAM blocks. After sending the read requests generated by one incoming spike, it transits to the idle state and waits for the next input spike.  The compute module receives the data requested by the load module from the SRAMs, updates the data using the saturated adder, and sends the results to the FIFO connected to the store module. The store module receives addresses of the loaded data from the load module and the results from the compute module. Using the data and addresses received, it writes the data back to the SRAMs. It also generates spikes when a condition, depending on the chosen techniques, TTFS or softmax, is met to be sent to the next layer. In addition, the core is designed based on the decoupled access-execute model <cit.>, which enables it to hide memory access latency by performing different computations parallelly. 



 §.§ Mapping


In this work, we plan to accelerate our trained SNN models on YOSO. To map a m × n fully connected SNN layer on YOSO, a minimum of C = MAX(n/N,m × n/W) PEs are needed where N is the maximum number of neurons that can be mapped to a single core and W is the maximum number of weights that the core can contain. The PEs are placed within a √(C) by √(C) grid. Each PE in the grid will be allocated to a layer or a part of layer for processing received by that layer.



 §.§ Hardware Simulation Methodology


We implemented the YOSO's NoC architecture using the OpenSMART NoC generator <cit.>. The X-Y routing mechanism is used to send the spike packets from one PE to another. In addition, we designed the PEs in such a way that each PE can meet the memory and computational requirements of at least 256 neurons <cit.>. To evaluate the performance of YOSO, we synthesized the hardware architecture shown in Fig. <ref> using Synopsys Design Compiler version P-2019.03-SP5 targeting a 22nm technology node with a 6×7 PE configuration. Gate-level simulations are performed using the Synopsys VCS-MX K-2015.09-SP2-9 and power analysis is performed using the Synopsys PrimePower version P-2019.03-SP5. 



§ EXPERIMENTS


In this section, we evaluate two SNNs: a fully connected SNN and a convolutional SNN on image classification task based on the MNIST dataset <cit.>, Fashion-MNIST dataset <cit.> and Caltech 101 face/motorbike dataset[http:
//www.vision.caltech.edu]. We benchmark their learning capabilities against existing spike-driven learning algorithms. In addition, we evaluate the power and energy efficiency of our fully connected SNN by accelerating its inference operations on YOSO.


 §.§ Temporal coding


 
We employ an efficient temporal coding scheme that encodes information as spiking time, with the assumption that strongly activated neurons tend to fire earlier <cit.>. 
 The input information is encoded in spike timing of neurons, with each neuron firing only once. More salient information is encoded as an earlier spike in the corresponding input neuron. The encoding spikes then propagate to  subsequent layers in a temporal fashion. Each neuron in the hidden and output layer receives the spikes from its presynaptic neurons, and emits a spike when the membrane potential reaches a threshold. Similar to the input layer, the neurons in the hidden and output layer that are strongly activated will fire first. Therefore, temporal coding is maintained throughout the DeepSNN, and the output neuron that fires earliest categorizes the input stimulus.




 §.§ MNIST Dataset


The MNIST dataset comprises of 60,000 28×28 grayscale images for training and 10,000 28×28 grayscale images for testing. We first convert the images into spike trains, the encoded spike patterns are trained with the fully connected SNN and convolutional SNN, respectively. The complete convolutional SNN model are shown in Fig. <ref>. Table <ref> shows the classification accuracies of the two SNNs, and other spike-driven learning algorithms on the MNIST dataset.
  





As shown in Table <ref>, the proposed STDBP learning algorithm could reach accuracies of 98.1% and 98.5% with network structures of 784-400-10 and 784-800-10, respectively. They outperform previously reported results with same network structure. For example, with the structure of 784-400-10, the classification accuracy of our method is 98.5%, while the accuracy achieved by Mostafa <cit.> is 97.5%. Another advantage of our algorithm is that it does not need additional training strategies, such as constraints on weights and gradient normalization, which are widely used in previous works to improve their performance <cit.>. This facilitates large-scale implementation of STDBP, and make it possible to train more complex CNN structure. The proposed convolutional SNN achieves an accuracy of is 99.4%, much higher than all the results obtained by the fully connected SNNs. To our best knowledge, this is the first implementation of a convolutional SNN structure with single-spike-timing-based supervised learning algorithms.

Fig. <ref> shows the distribution of spike timing in the hidden layers and of the earliest spike time in the output layer across 10000 test images for two SNNs, namely 784-400-10 and 784-800-10.  For both architectures, the SNN makes a decision after only a fraction of the hidden layer neurons. For the 784-400-10 topology, an output neuron spikes (a class is selected) after only 48.6% of the hidden neurons have spiked. The network is thus able to make very rapid decisions about the input class. In addition, during the simulation time, only 66.3% of the hidden neurons have spiked. Therefore, the experimental results demonstrate that the proposed learning algorithm works in a accurate, fast and sparse manner.

To investigate whether the proposed ReL-PSP solves the problems of gradient exploding and dead neuron, we take the fully connected SNNs as an example and 20 independent experiments are conducted with different initial synaptic weights.
 The values of gradient and the number of dead neurons are reported in Fig. <ref> and Fig. <ref>.
 
 
 

Fig. <ref>(a) and (b) show the distribution of spike gradients of the ReL-PSP and the alpha-PSP function, respectively. As shown in Fig. <ref>(a), there are some spike gradients is near 0, which will cause the gradient exploding problem. Previous attempts to alleviate this problem, such as adaptive learning rate <cit.> and dynamic firing threshold <cit.>, can not fully resolve it. Fig. <ref>(b) shows the spike gradients distribution of the proposed ReL-PSP function. The values of spike gradients are far away from 0 to prevent the gradient exploding problem. Therefore, stable gradient propagation in DeepSNNs can be achieved by the proposed ReL-PSP function. 



 Fig. <ref> shows the trend of the number of dead neurons along the learning process with different postsynaptic potential functions. As learning progresses, the number of the dead neurons increases in both SNNs. In neural networks, typically only a subset of neurons are activated <cit.> in action. However, excessive inactivated neurons lead to the dead neuron problem, which reduces effective capacity of the network and affects the predictive performance. As we mentioned in <ref>-A, due to the leaky nature of traditional PSP function and the spike generation mechanism, SNNs are more likely to suffer from the dead neuron problem. As shown in Fig. <ref>, after 100 learning epochs, the SNN with ReL-PSP has 30% active neurons and can achieve an accuracy of 98.4% on the test dataset. However, the SNN with alpha-PSP only has 5% active neuron with an accuracy of 92.5%. The experimental results corroborate our hypothesis that traditional PSP functions suffer from dead neuron problem, that limits the network predictive performance.





 §.§ Fashion-MNIST dataset

Fashion-MNIST dataset <cit.> has same number of training/testing samples as MNIST, but it is  more challenging than MNIST. The samples are associated with 10 different classes like T-shirt/top, Pullover, Trouser, Dress, Sandal, Coat, Shirt, Bag,
Sneaker and Ankle boot. In this part, we use it to compare our method with the existing fully-connected feedforward SNNs and convolutional SNNs. 





Table <ref> shows the classification accuracies and characteristics of different methods on Fashion-MNIST dataset. The proposed learning algorithm still deliver the best test accuracy in temporal coding based SNN methods. For example, our method can achieve accuracies of 88.1% and 90.1% with the fully-connected SNN and convolutional SNN, respectively. These results outperforms the best reported result of 88.0%  <cit.> in temporal coding based SNN models.



 §.§ Caltech face/motorbike dataset


In this experiment, the performance of the proposed STDBP is evaluated on the face/motobike categories of the Caltech 101 dataset (http://www.vision.caltech.edu).  For each category, the training and validation set consist of 200 and 50 randomly selected samples, respectively, and the others are regard as testing samples. Before training, all images are rescaled and converted to 160×250 grayscale images. Fig. <ref> shows some samples of the converted images. The converted images are then encoded into spike patterns by temporal coding.





The classification accuracies of different SNN-based computational models are shown in Table <ref>. The proposed STDBP learning algorithm achieves an accuracy of 99.2% with the fully connected SNN structure and an accuracy of 99.5% with the convolutional SNN structure. The accuracy obtained by STDBP outperforms the previously reported SNN-based methods on this dataset. For example, In Kheradpisheh et al <cit.>, an convolutional SNN structure with a SVM classifier achieves an accuracy of 99.1% on the same dataset. Moreover, it is not a fully spike-based computational model that the membrane potential is used as the classification signal. Recently, a spike-based fully connected SNNs model achieves an accuracy of 99.2%. However, the proposed method with convolutional SNN structure reaches an accuracy of 99.5%, which is the state-of-the-art performance on this benchmark.



 §.§ Hardware Simulation Results



We will now focus on evaluating the power and energy efficiency of our SNN models by accelerating their inference operations on YOSO. As a baseline for comparison with other neuromorphic accelerators, we considered a fully connected SNN with network architecture 784-800-10 and trained it on MNIST data with proposed STDBP learning algorithm. The learned weights are then transferred to YOSO for accelerating the inference operations. As shown in Table <ref>, YOSO consumes 0.751 mW of total power consumption and 47.71 ms of latency to classify an image from MNIST dataset. In addition, YOSO achieves 399×, 159.8×, 143.8×, and 1.67× power savings as compared to Spinnaker <cit.>, Tianji <cit.>, TrueNorth-b <cit.>, and Shenjing <cit.>, respectively. Though, TrueNorth-a consumes less power than YOSO, there is a significant difference between the classification accuracies of TrueNorth-a and YOSO (See Table <ref>). Moreover, YOSO provides 108.7×, 6×, and 3× energy efficiency as compared to Spinnaker <cit.>, SNNwt <cit.>, and TrueNorth-b <cit.>, respectively.



§ DISCUSSION AND CONCLUSION


In this work, we analysed the problems that BP faces in a DeepSNN, namely the non-differentiable spike function, the exploding gradient, and the dead neuron problems. To address these issues, we proposed the Rectified Linear Postsynaptic Potential function (ReL-PSP) for spiking neurons and the STDBP learning algorithm for DeepSNNs. We evaluated the proposed method on both a multi-layer fully connected SNN and a convolutional SNN. Our experiments on MNIST showed an accuracy of 98.5% in the case of the the fully connected SNN and 99.4% with the convolutional SNN, which is the state-of-art in spike-driven learning algorithms for DeepSNNs. 

There have been a number of learning algorithms for DeepSNNs, such as conversion methods  <cit.>, and surrogate gradients methods <cit.>. These methods are not compatible with temporal coding and spike-based learning mechanism. 
To overcome the non-differentiability of spike function, many methods have been studied <cit.> to facilitate backpropagation. Two common drawback of these methods are exploding gradients and dead neurons, which have been partially addressed using techniques such as constraints on weights and gradient normalization. These techniques affect learning efficiency, thus limiting the scope of their applications in large-scale networks. The proposed STDBP learning algorithm with ReL-PSP spiking neuron model can train DeepSNNs directly without any additional technique, hence allowing the DeepSNNs to scale, as shown in the high accuracy of the convolutional SNN. 

In addition to being fast, sparse and more accurate, the proposed ReL-PSP neuron model and STDBP have other attributes that might make it more energy-efficient and (neuromorphic) hardware friendly. Firstly, the linear ReL-PSP function is simpler than alpha-PSP for hardware implementation. Secondly, unlike rate-based encoding methods that require more time to generate enough output spikes for classification, our method takes advantage of temporal coding and uses a single spike, which is more sparse and energy-efficient, given energy is mainly consumed during spike generation and transmission. Thirdly, without additional training techniques, on-chip training in neuromorphic chips would be much easier to realize. Finally, even if the training is performed offline, inference of our SNN models can be accelerated with much less power and energy consumption as compared to other state-of-the-art neuromorphic accelerators. 

IEEEtran 
 



