As shown in Figure , the input layer has the same number of units as the dimension of the input vector (i.e. 𝐷-dimensional in this case). It takes in the input 𝒙 and passes it to the first hidden layer. Each two consecutive hidden layers are connected by a set of weights and biases along with an activation function. Specifically, let us denote the 𝑖 𝑡ℎ pre-activation and post-activation unit in the 𝑙 𝑡ℎ layer as 𝑧 𝑖 𝑙 and 𝑥 𝑖 𝑙 , respectively. Also, we denote the weights and biases in the 𝑙 𝑡ℎ layer as 𝑊 𝑖𝑗 𝑙 and 𝑏 𝑖 𝑙 , respectively.