title: Is there a way to add keras 'custom layer' based/specific penalty to the overall loss function?
body: <p>i have a keras sequential model with some custom layers in it. Now in one of the layers, based on the input of that specific layer, i want to calculate a penalty and i want the penalty to be added to the loss function which the optimizer tries to minimize overall. </p>
  
  <p>I have gone through the concept of <code>tf.keras.layers.ActivityRegularization</code> but struggling to figure out how to solve my issue.</p>