title: Keras loading model issue
body: <p>So I'm having an issue and I'm stuck on it for a while now</p>
  
  <p>I'm trying to understand and repeat a jupyter tutorial about adversarial example.
  However, when I'm loading a model, depending on how is it done I have big differences between the two models accuracy:
    - when it's loading inside a class, there is no issues
    - when it's loading on jupyter, the accuracy decrease</p>
  
  <p>The class use the following lines:</p>
  
  <pre><code>from keras.models import Sequential, load_model
  
  class Lenet:
  
    def __init__(self, epochs=200, batch_size=128, load_weights=True):
      self.name               = 'lenet'
      self.model_filename     = 'networks/models/lenet.h5'
      self.num_classes        = 10
      self.input_shape        = 32, 32, 3
      self.batch_size         = batch_size
      self.epochs             = epochs
      self.iterations         = 391
      self.weight_decay       = 0.0001
      self.log_filepath       = r'networks/models/lenet/'
  
      if load_weights:
          try:
              self._model = load_model(self.model_filename)
              print('Successfully loaded', self.name)
          except (ImportError, ValueError, OSError):
              print('Failed to load', self.name)
  </code></pre>
  
  <p>what I am doing is:</p>
  
  <pre><code>modelPath = "networks/models/lenet.h5"
  lenet2 = keras.models.load_model(modelPath)
  
  modelPath2 = "networks/models/resnet.h5"
  resnet2 = keras.models.load_model(modelPath2)
  </code></pre>
  
  <p>However, when I'm testing the accuracy of the two models:</p>
  
  <pre><code>lenet = LeNet()
  resnet = ResNet() #the class is nearly the same as LeNet
  models = [lenet, lenet2, resnet, resnet2]
  network_stats, correct_imgs = helper.evaluate_models(models, x_test, y_test)
  network_stats = pd.DataFrame(network_stats, columns=['name', 'accuracy', 'param_count'])
  </code></pre>
  
  <p>I'm getting this following result: (network_stats)</p>
  
  <pre><code>name                accuracy    param_count
  0   lenet           0.7488            62006
  1   sequential_1    0.4800            62006
  2   resnet          0.9231           470218
  3   model_1         0.1092           470218
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/aY3H3.jpg" rel="nofollow noreferrer">link to the  picture</a></p>
  
  <p>to explain the picture: on the left, <a href="https://github.com/Hyperparticle/one-pixel-attack-keras/blob/master/networks/lenet.py" rel="nofollow noreferrer">this is the class lenet</a>, that give good results. On the right, my failed try to load the same model</p>
  
  <p>Do you have any idea about why is that happening?</p>
  
  <p>--edit--</p>
  
  <p>can't post images yet on stackoverflow, so I explained a little bit more the issue</p>
  
  <p>--edit2--</p>
  
  <p>can reproduce with another network (resnet)</p>