title: Create a 3 layer neural network so that weight matrix is orthogonal?
body: <p>I was asked in an interview question and I am not sure if this was even a valid question but if this is somehow, can anyone please let me know the answer. This might be little subjective as it is related to architecture but I didn't where else to post this question.</p>
  
  <p>The question is as follows.</p>
  
  <p>You are given a dataset with a class label. You need to create a 3 layers network with one hidden layer and one input and one output layer. What changes you would make in terms of the architecture of the network so that the weight matrix of the network stays orthogonal.</p>
  
  <p>It should be irrespective of the given dataset. </p>