title: Backpropagation for sigmoid activation and softmax output
body: <p>I am trying to build a L layer neural network for multi-class classification with softmax activation in the output layer and sigmoid activation in other layers. </p>
  
  <p>The function used for training looks like this:</p>
  
  <pre><code>def L_layer_model(X, Y, layers_dims, learning_rate=0.01, num_iterations=5000, print_cost=True): 
  """
  Implements a L-layer neural network: [LINEAR-&gt;SIGMOID]*(L-1)-&gt;LINEAR-&gt;SOFTMAX.
  
  Arguments:
  X -- data, numpy array of shape (number of features, number of examples)
  Y -- true "label" vector of shape (number of classes, number of examples)
  layers_dims -- list containing the input size and each layer size, of length (number of layers + 1).
  learning_rate -- learning rate of the gradient descent update rule
  num_iterations -- number of iterations of the optimization loop
  print_cost -- if True, it prints the cost every 100 steps
  
  Returns:
  parameters -- parameters learnt by the model. They can then be used to predict.
  """
  
  np.random.seed(1)
  costs = []                         # keep track of cost
  
  # Parameters initialization.
  
  parameters = initialize_parameters_deep(layers_dims)
  L = len(parameters) // 2   # number of layers in the neural network
  forward_calculated = {}
  m = Y.shape[1]
  
  
  # Loop (gradient descent)
  for i in range(0, num_iterations):
  
      # Forward propagation: [LINEAR -&gt; SIGMOID]*(L-1) -&gt; LINEAR -&gt; SOFTMAX.
      A = X
      forward_calculated["A0"] = X
      for l in range(1, L+1):   
          A_prev = A
          #print(A_prev)
  
          W = parameters['W' + str(l)]
          b = parameters['b' + str(l)]
          #print("W.shape: "+str(W.shape))
          #print("A_prev.shape: "+str(A_prev.shape))
          #print("b.shape: "+str(b.shape))
          Z = np.dot(W, A_prev) + b
          #Z = np.matmul(W, A_prev) + b
  
          assert(Z.shape == (W.shape[0], A.shape[1]))
          forward_calculated["Z" + str(l)] = Z   # store for future use
  
          if l != L:  # except the last layer
              A = sigmoid(Z)
          else:
              A = softmax(Z)
  
          #print("A is a tuple: ", end='')
          #print(isinstance(A, tuple))
  
          forward_calculated["A" + str(l)] = A   # store for future use
  
      assert(forward_calculated["A" + str(L)].shape == (NUMBER_OF_CLASSES, X.shape[1]))
  
  
      # Compute cost.
      Y_hat = forward_calculated["A" + str(L)]
      cost = compute_multiclass_loss(Y, Y_hat)
  
      #cost = compute_cost(AL, Y)
  
      # now back propagation
  
      grads = {}
      grads['dZ' + str(L)] = forward_calculated["A" + str(L)] - Y
      grads['dW' + str(L)] = (1./m) * np.dot(grads['dZ' + str(L)], forward_calculated["A" + str(L-1)].T)
      grads['db' + str(L)] = (1./m) * np.sum(grads['dZ' + str(L)], axis=1, keepdims=True)
  
      for l in range(L-1, 0, -1):
          grads['dA' + str(l)] = np.dot(parameters["W" + str(l+1)].T, grads['dZ' + str(l+1)])
          #dA1 = np.matmul(W2.T, dZ2)
          grads['dZ' + str(l)] = grads['dA' + str(l)] * sigmoid(forward_calculated["Z" + str(l)]) * (1 - sigmoid(forward_calculated["Z" + str(l)]))
          #dZ1 = dA1 * sigmoid(Z1) * (1 - sigmoid(Z1))
          grads['dW' + str(l)] = (1./m) * np.dot(grads['dZ' + str(l)], forward_calculated["A" + str(l-1)].T)
          #dW1 = (1./m) * np.matmul(dZ1, X.T)
          grads['db' + str(l)] = (1./m) * np.sum(grads['dZ' + str(l)], axis=1, keepdims=True)
          #db1 = (1./m) * np.sum(dZ1, axis=1, keepdims=True)
  
  
      # Update parameters.
  
      for l in range(1,L+1):
          #print("grads[dW]: " + str(grads["dW" + str(l)]));
          parameters["W" + str(l)] = parameters["W" + str(l)] - learning_rate * grads["dW" + str(l)]
          #print("grads[db]: " + str(grads["db" + str(l)]));
          parameters["b" + str(l)] = parameters["b" + str(l)] - learning_rate * grads["db" + str(l)]
  
      # Print the cost every 100 training example
      if print_cost and i % 100 == 0:
          print ("Cost after iteration %i: %f" % (i, cost))
  
          costs.append(cost)
  
  print(costs)
  # plot the cost
  plt.plot(np.squeeze(costs))
  plt.ylabel('cost')
  plt.xlabel('iterations (per tens)')
  plt.title("Learning rate =" + str(learning_rate))
  plt.show()
  
  return parameters
  </code></pre>
  
  <p>When I have only one hidden layer, the code works fine and the model gradually converges. However, when I have more than one hidden layer, the model doesn't seem to converge. It predicts all the examples to be of the same class. Is there any error in my formula of back propagation? The cost function I am using is the log loss.</p>
  
  <pre><code>def compute_multiclass_loss(Y, Y_hat):   # Y -&gt; actual, Y_hat -&gt; predicted
  
  L_sum = np.sum(np.multiply(Y, np.log(Y_hat)))
  m = Y.shape[1]
  L = -(1/m) * L_sum
  
  L = np.squeeze(L)      # To make sure your cost's shape is what we expect (e.g. this turns [[17]] into 17).
  assert(L.shape == ())
  return L
  </code></pre>
  
  <p>In short, my question is, are these formula true (for log loss and softmax activation at the output and sigmoid activation at other layers)?
  <a href="https://i.stack.imgur.com/KjLyq.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/KjLyq.png" alt="enter image description here"></a></p>