title: Very large loss values when training multiple regression model in Keras
body: <p>I was trying to build a multiple regression model to predict housing prices using the following features:</p>
  
  <pre><code>  [bedrooms  bathrooms  sqft_living  view  grade]
  = [0.09375   0.266667     0.149582   0.0    0.6]
  </code></pre>
  
  <p>I have standardized and scaled the features using <code>sklearn.preprocessing.MinMaxScaler</code>.</p>
  
  <p>I used Keras to build the model:</p>
  
  <pre class="lang-py prettyprint-override"><code>def build_model(X_train):
      model = Sequential()
      model.add(Dense(5, activation = 'relu', input_shape = X_train.shape[1:]))
          model.add(Dense(1))
  
      optimizer = Adam(lr = 0.001)
  
      model.compile(loss = 'mean_squared_error', optimizer = optimizer)
  
      return model
  </code></pre>
  
  <p>When I go to train the model, my loss values are insanely high, something like 4 or 40 trillion and it will only go down about a million per epoch making training infeasibly slow. At first I tried increasing the learning rate, but it didn't help much. Then I did some searching and found that others have used a log-MSE loss function so I tried it and my model seemed to work fine. (Started at 140 loss, went down to 0.2 after 400 epochs)</p>
  
  <p><strong>My question is do I always just use log-MSE when I see very large MSE values for linear/multiple regression problems? Or are there other things i can do to try and fix this issue?</strong> </p>
  
  <p>A guess as to why this issue occurred is the scale between my predictor and response variables were vastly different. <code>X</code>'s are between 0-1 while the highest <code>Y</code> went up to 8 million. (Am I suppose to scale down my <code>Y</code>'s? And then scale back up for predicting?)</p>