Ram usage of model keep increasing when training to previously trained model
I found it and answering as it might help somebody.

I saw pattern in this problem. Every time I load the model from my saved ones and then start them training again it will start eating my ram. Slowly it will reach to limit of my laptop.
But same model if i train from scratch will not do same. 
I was loading model using

tf.keras.models.load_model("models/min__1576172002.model")


So Instead of training it again. I Build a new one and assign its weights = trained models weight. This way this problem is solved.

        prev_modal = tf.keras.models.load_model("models/min__1576224504.h5")
        model = Sequential()
        model.add(Dense(6,activation='relu',input_shape=(4,)))
        model.add(Dropout(0.2))
        # model.add(Flatten())

        model.add(Dense(5,activation='relu'))
        model.add(Dropout(0.2))
        # model.add(Flatten())

        model.add(Dense(5,activation='relu'))
        model.add(Dropout(0.2))


        model.add(Dense(2,activation='softmax'))

        model.compile(loss="mse", optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])

        model.set_weights(prev_modal.get_weights())