Does matrix shape for training/testing sets have to be in a particular order?
In practice

Andrew Ng seems to be using the convention on the Theano framework. If you had 10 colour images, each 100 pixels high and 200 pixels wide, Theano models would expect input of the form:


  (batch size, input channels, input rows, input columns)


and so dimensions: (10, 3, 100, 200). The three is because of the three RGb dimensions in a colour image.

Tensorflow on the other hand reverse this order to instead use:
(num_obs, height, width, channels) - for the same example as above this becomes:


  (batch size, input rows, input columns, input channels)


meaning the dimensions of the input should be (10, 100, 200, 3).

Keras works with Tensorflow and Theano and supports both conventions by simply allowing the user to set the position in which the number of channels are given. This can be set in a config file or using a specific environment variable upon setup. Have a look at the relevant documentation. 



(Possible) justifications

There are different points of view on what is more natural. If you come from a computer vision arena (or image processing in general), then libraries like OpenCV use a coordinate system that has (0,0) in the top left of an image, and you specify a single pixel by given the vertical movement, then the horizontal movement from the origin. This means you give a height, then a width. One justification for this might be that many computer vision algorithms (e.g. colour filters) focus on difference across the channels of an image, but it is just convention.

In general graphing and mathematics, it is common to give the X coordinates first, then the Y, which means talking about horizontal movements, then vertical. Linear algebra in general is a good example of this.

All this being said, the best advice is to make sure you use the correct dimensions according to the documentation of the software you use. If you make your own software - you get to choose! (But make sure to document it!)