What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?
I don't think there's a &quot;standard way&quot; of expressing the forward pass: you use the transpose when you need to use it, and this depends on how you define the weights and inputs matrices, and on the architecture of your neural network. For example, in a fully connected feedforward neural network, you know that every neuron in the previous layer is connected to every neuron in the current layer, so, as long as this is satisfied when you multiply the matrices, it does not matter whether you use transposes or not, and I don't think that, in computational terms, it makes any difference if you use transposes or not. (By the way, if you are writing something, I suggest that you always specify the dimensions of your matrices and your conventions).
Of course, if you want to use a library like TensorFlow, you will probably need to follow the conventions of the library, but this is another story.