title: Should you normalize outputs of a neural network for regression tasks?
body: <p>I've made a CNN that takes a signal as input and outputs the parameters used in a simulation to create that signal. I've heard that for regression tasks <a href="https://stackoverflow.com/questions/42604261/neural-networks-normalizing-output-data">you don't normally normalize the outputs</a> to a neural network. But the variables the model is trying to predict have very different standard deviations, like one variable is always in the range of [1x10^-20, 1x10-24] while another is almost always in the range of [8, 16]. My question is since all loss functions first take the difference between the target and actual output values and this difference would naturally scale with the std of that output variable wouldn't loss of the network mostly dependent on the accuracy of the output variables with large stds and not ones with small stds? Also would unnormalized output hinder the training process since the network can get low loss for an output variable with very low std by just guessing values close to its mean?</p>
  
  <p>If this is the case why can't I find much on the internet talking about or suggesting to normalize outputs? It seems really important for getting reliable loss values.</p>