overfit random walk using ANN in Keras I am trying to build a neural net that will overfit random walk path. So, far I wasn't able to get a neural net that we shatter/overfit. I was wondering which parameters I should explore, or which guide lines I should follow to achieve this (wierd) task.
Code:
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import SGD,Adam
from keras import regularizers
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline  
import random
import math

model=Sequential()
num_units=100
model.add(Dense(num_units,input_shape=(1,),activation='tanh')) 
model.add(Dense(num_units,activation='tanh'))
model.add(Dense(num_units,activation='tanh'))
model.add(Dense(num_units,activation='tanh'))
model.add(Dense(1, activation='tanh')) #output layer 1 unit
model.compile(Adam(),'mean_squared_error',metrics=['mse'])

num_of_steps=3000;
step_size=0.01
x_all=list(range(0,num_of_steps))

##########################random walk functino
random_walk=[0]
for i in x_all[1:]:
    f=np.random.uniform(0,1,1)
    if f&lt;0.5:
        random_walk.append(random_walk[i-1]+step_size)
    else:
        random_walk.append(random_walk[i-1]-step_size)
########################

x1=list(range(1,1000,2))
x2=list(range(2,1000,2))
y1=[random_walk[x] for x in x1]
y2=[random_walk[x] for x in x2]

model.fit(x1,y1,epochs=2500,verbose=1)
fit1=model.predict(x1)
fit2=model.predict(x2)
plt.plot(x1,y1,'k')
plt.plot(x2, y2, 'r')
plt.scatter(x1, fit1, facecolors='none', edgecolors='g') #plt.plot(x_value,sample,'bo')
plt.scatter(x2, fit2, facecolors='none', edgecolors='b') #plt.plot(x_value,sample,'bo')

this is an example of the results I am getting:

And I would want to get a better fit (to my sample data: x1,y1).
Try changing your kernel regularizer. Either reduce its effect or shut it off entirely, then retrain and see what happens.

EDIT: I expect that the kernel regularizer is your best bet. You could also experiment with changing the depth of the network (number of layers) and how many units you give each layer. I would expect if you put more units into the layer that currently has 10, you'd see more overfitting.