How can I have the same initialization for all my networks? I want to have the same weights for layer initializations in all my networks, so that when I'm comparing their first epoch loss they all start from the same value. Is there a way in keras to do this? 

I have set the random seed for the numpy and tensorflow, but still I get different results in initializations.
You need to specify the seed in the initializer, e.g:

from keras.initializers import RandomUniform

seed = 0

model.add(Dense(64, kernel_initializer = RandomUniform(minval = -0.05, 
                                                       maxval =  0.05, 
                                                       seed = seed)))