title: How to overfit data with Keras?
body: <p>I'm trying to build a simple regression model using keras and tensorflow. In my problem I have data in the form <code>(x, y)</code>, where <code>x</code> and <code>y</code> are simply numbers. I'd like to build a keras model in order to predict <code>y</code> using <code>x</code> as an input.</p>
  
  <p>Since I think images better explains thing, these are my data:</p>
  
  <p><a href="https://i.stack.imgur.com/npCjz.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/npCjz.jpg" alt="enter image description here"></a></p>
  
  <p>We may discuss if they are good or not, but in my problem I cannot really cheat them.</p>
  
  <p>My keras model is the following (data are splitted 30% test <code>(X_test, y_test)</code> and 70% training <code>(X_train, y_train)</code>):</p>
  
  <pre><code>model = tf.keras.Sequential()
  
  model.add(tf.keras.layers.Dense(32, input_shape=() activation="relu", name="first_layer"))
  model.add(tf.keras.layers.Dense(16, activation="relu", name="second_layer"))
  model.add(tf.keras.layers.Dense(1, name="output_layer"))
  
  model.compile(loss = "mean_squared_error", optimizer = "adam", metrics=["mse"] )
  
  history = model.fit(X_train, y_train, epochs=500, batch_size=1, verbose=0, shuffle=False) 
  eval_result = model.evaluate(X_test, y_test)
  print("\n\nTest loss:", eval_result, "\n")
  
  predict_Y = model.predict(X)
  </code></pre>
  
  <p>note: <code>X</code> contains both <code>X_test</code> and <code>X_train</code>.</p>
  
  <p>Plotting the prediction I get (blue squares are the prediction <code>predict_Y</code>)</p>
  
  <p><a href="https://i.stack.imgur.com/KKiRd.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/KKiRd.png" alt="enter image description here"></a></p>
  
  <p>I'm playing a lot with layers, activation funztions and other parameters. My goal is to find the best parameters to train the model, but the actual question, here, is slightly different: in fact I have hard times to force the model to overfit the data (as you can see from the above results).</p>
  
  <p>Does anyone have some sort of idea about how to reproduce overfitting?</p>
  
  <p>This is the outcome I would like to get:
  <a href="https://i.stack.imgur.com/4IvqX.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/4IvqX.png" alt="enter image description here"></a></p>
  
  <p>(red dots are under blue squares!)</p>
  
  <p>EDIT:</p>
  
  <p>Here I provide you the data used in the example above: you can copy paste directly to a python interpreter:</p>
  
  <pre><code>X_train = [0.704619794270697, 0.6779457393024553, 0.8207082120250023, 0.8588819357831449, 0.8692320257603844, 0.6878750931810429, 0.9556331888763945, 0.77677964510883, 0.7211381534179618, 0.6438319113259414, 0.6478339581502052, 0.9710222750072649, 0.8952188423349681, 0.6303124926673513, 0.9640316662124185, 0.869691568491902, 0.8320164648420931, 0.8236399177660375, 0.8877334038470911, 0.8084042532069621, 0.8045680821762038]
  y_train = [0.7766424210611557, 0.8210846773655833, 0.9996114311913593, 0.8041331063189883, 0.9980525368790883, 0.8164056182686034, 0.8925487603333683, 0.7758207470960685, 0.37345286573743475, 0.9325789202459493, 0.6060269037514895, 0.9319771743389491, 0.9990691225991941, 0.9320002808310418, 0.9992560731072977, 0.9980241561997089, 0.8882905258641204, 0.4678339275898943, 0.9312152374846061, 0.9542371205095945, 0.8885893668675711]
  X_test = [0.9749191829308574, 0.8735366740730178, 0.8882783211709133, 0.8022891400991644, 0.8650601322313454, 0.8697902997857514, 1.0, 0.8165876695985228, 0.8923841531760973]
  y_test = [0.975653685270635, 0.9096752789481569, 0.6653736469114154, 0.46367666660348744, 0.9991817903431941, 1.0, 0.9111205717076893, 0.5264993912088891, 0.9989199241685126]
  X = [0.704619794270697, 0.77677964510883, 0.7211381534179618, 0.6478339581502052, 0.6779457393024553, 0.8588819357831449, 0.8045680821762038, 0.8320164648420931, 0.8650601322313454, 0.8697902997857514, 0.8236399177660375, 0.6878750931810429, 0.8923841531760973, 0.8692320257603844, 0.8877334038470911, 0.8735366740730178, 0.8207082120250023, 0.8022891400991644, 0.6303124926673513, 0.8084042532069621, 0.869691568491902, 0.9710222750072649, 0.9556331888763945, 0.8882783211709133, 0.8165876695985228, 0.6438319113259414, 0.8952188423349681, 0.9749191829308574, 1.0, 0.9640316662124185]
  Y = [0.7766424210611557, 0.7758207470960685, 0.37345286573743475, 0.6060269037514895, 0.8210846773655833, 0.8041331063189883, 0.8885893668675711, 0.8882905258641204, 0.9991817903431941, 1.0, 0.4678339275898943, 0.8164056182686034, 0.9989199241685126, 0.9980525368790883, 0.9312152374846061, 0.9096752789481569, 0.9996114311913593, 0.46367666660348744, 0.9320002808310418, 0.9542371205095945, 0.9980241561997089, 0.9319771743389491, 0.8925487603333683, 0.6653736469114154, 0.5264993912088891, 0.9325789202459493, 0.9990691225991941, 0.975653685270635, 0.9111205717076893, 0.9992560731072977]
  </code></pre>
  
  <p>Where <code>X</code> contains the list of the x values and <code>Y</code> the corresponding y value. (X_test, y_test) and (X_train, y_train) are two (non overlapping) subset of (X, Y).</p>
  
  <p>To predict and show the model results I simply use matplotlib (imported as plt):</p>
  
  <pre><code>predict_Y = model.predict(X)
  plt.plot(X, Y, "ro", X, predict_Y, "bs")
  plt.show()
  </code></pre>