title: Why Keras behave better than Pytorch under the same network configuration?
body: <p>Recently,  I have compared unet++ implementation of Keras version and Pytorch version on the same dataset. However, with Keras the loss decrease continuously and the accuracy is higher after 10 epochs, while with Pytorch the loss decrease unevenly and the  accuracy is lower after 10 epochs. Anyone has met such problems and has any answers?</p>
  <p>the final pytorch training process is like:</p>
  <blockquote>
  <p>2019-12-15 18:14:20 Epoch:9 Iter: 1214/1219  loss:0.464673 acc:0.581713</p>
  <p>2019-12-15 18:14:21 Epoch:9 Iter: 1215/1219  loss:0.450462 acc:0.584101</p>
  <p>2019-12-15 18:14:21 Epoch:9 Iter: 1216/1219  loss:0.744811 acc:0.293406</p>
  <p>2019-12-15 18:14:22 Epoch:9 Iter: 1217/1219  loss:0.387612 acc:0.735630</p>
  <p>2019-12-15 18:14:23 Epoch:9 Iter: 1218/1219  loss:0.767146 acc:0.364759</p>
  </blockquote>
  <p>the final keras training process is like:</p>
  <blockquote>
  <p>685/690 [============================&gt;.] - ETA: 2s - loss: 0.4940 - acc:  0.7309</p>
  <p>686/690 [============================&gt;.] - ETA: 1s - loss: 0.4941 - acc: 0.7306</p>
  <p>687/690 [============================&gt;.] - ETA: 1s - loss: 0.4939 - acc:  0.7308</p>
  <p>688/690 [============================&gt;.] - ETA: 0s - loss: 0.4942 - acc:  0.7303</p>
  <p>689/690 [============================&gt;.] - ETA: 0s - loss: 0.4943 - acc: 0.7302</p>
  </blockquote>