Keras customizing train_step when using a custom optimizer
Ops, I was using different API's for my optimizer and my model, one was using keras, the other was using tensorflow.keras. If anyone runs into similar issues this might be the problem.
I'm trying to change what happens in train_step when using a custom optimizer but I get this error message when trying to compile the model: 

raise ValueError('Could not interpret optimizer identifier:', identifier) ValueError: ('Could not interpret optimizer identifier:', <main.Optimizer_quazi_Newton object at 0x0000019CFCC2EF60>)

The problem I'm having is that I want to selectively be able to feed the same data during training depending on what happens in the optimizer. My optimizer has a variable which basically say "I want the same data again" and I want to modify the fit function so that when the optimizer needs it I can feed it the same data instead of going to the next batch.