Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies
First, try to understand a few points -

Output Neuron value and the prediction both are the same things. In the case of Classification, we convert the output probability to Class based on a Threshold.

MSE is used in Regression and In a regression problem, you mostly have one output Neuron e.g. Price. You may have more if you want to club multiple targets e.g. Bounding Box problem


The &quot;N&quot; in the denominator is the number of different errors calculated which is basically the number of samples in context.
With &quot;In context&quot;, I meant if it's at the time of backpropagation it is the batch_size. if it's at the end of epoch/training it's the whole training dataset.

what does $h_θ(x^i)$ represent?

It is the prediction(value of output Neuron) for $i_{th}$ sample.