title: Keras high loss, not decreasing with each epoch
body: <p>I am learning NN and Keras. My test data is something like this:</p>
  
  <pre><code>Result, HomeWinPossibility, DrawPossibility, AwayWinPossibility
  [['AwayWin' 0.41 0.28 0.31]
   ['HomeWin' 0.55 0.25 0.2]
   ['AwayWin' 0.17 0.21 0.62]
   .....
  </code></pre>
  
  <p>Here is my model:</p>
  
  <pre class="lang-py prettyprint-override"><code>model = Sequential()
  model.add(Dense(16, input_shape=(3,)))
  model.add(Activation('sigmoid'))
  model.add(Dense(8, activation='relu'))
  model.add(Dense(3))
  model.add(Activation('softmax'))
  model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=["accuracy"])
  model.fit(train_X, train_y_ohe, epochs=100, batch_size=1, verbose=1);
  </code></pre>
  
  <p>The output from fit is:</p>
  
  <pre><code>Epoch 1/100
  190/190 [==============================] - 1s 3ms/step - loss: 0.9151 - acc: 0.5737
  Epoch 2/100
  190/190 [==============================] - 1s 3ms/step - loss: 0.9181 - acc: 0.5474
  Epoch 3/100
  190/190 [==============================] - 1s 3ms/step - loss: 0.9111 - acc: 0.5526
  Epoch 100/100
  190/190 [==============================] - 1s 3ms/step - loss: 0.9130 - acc: 0.5579
  </code></pre>
  
  <p>So why the loss is not going down as some NN tutorials I read? Is it because the data I provided are just noises, so NN can't find any clue or something not right with my model?</p>
  
  <p>As the <code>acc</code> is always around 0.55 (so 50%), does it mean the NN actually achieved better than random guessing (> 33%)? If this is true, why it achieved accuracy 0.57 at the first epoch?</p>