How to calculate change in Loss Function w.r.t Weights for a fully connected NN I read the article Optimal Brain Damage by Yann LeCun, John S. Denker and Sara A. Solla where the authors discuss estimating the saliency of each weight of a neural network (which they define by the change in the loss function upon perturbing the weight). The larger the saliency, the more effect the weight has on the learning.
Is there any code/API in Keras or Tensorflow to calculate such &quot;weight saliency&quot; for a simple fully connected neural network (with one or two hidden layers)?
I doubt there is any ready-to-use facility but you can achieve this very easily with some custom code.
You can do -

Train your model(to the extent you want)
Get output using predict method on a dataset
Calculate the Loss using Loss function, y_true and last step output
Get the weight of every layer using Keras get_weights() API
Update weight as per your experiment using Keras set_weights()  API
Calculate the new Loss using the same process as mentioned in step#3

Keras weights API -
You can get weights as a simple array and can update accordingly.
May traverse in the loop on Layers Or access directly using array index.
Some code snippets(See reference links for detail)
Layer.get_weights()
Layer.set_weights(weights)

for layer in model.layers:
    print(layer .get_weights())

weights = model.layers[1].get_weights()[0]
biases = model.layers[1].get_weights()[1]

weights = [np.random.rand(*w.shape) for w in model.get_weights()]
# update
model.set_weights(weights)

References
Keras.IO
Stackoverflow
Stackoverflow
Stackoverflow