How can I measure activation level of each nodes when predict using trained model?
I'm interesting in which nodes are activated and how much strongly activated when some input is entered to trained model.
The picture as bellow shows what I want to get from model.
(I want to know the degree of activation in every nodes)
As I know there are some techniques to visualize what the nodes(or filter) are paying attention to. (especially in CNN)
Is there any good way to measure how active each nodes are?
I usually using Keras. But pyTorch is ok too.


You are looking for activation map / grad cam ..., you can get a look at : https://keras.io/examples/vision/grad_cam/ and https://keras.io/examples/vision/visualizing_what_convnets_learn/
You can also try keract : https://github.com/philipperemy/keract
There are others github repository on the subject on explainability in the AI field, such as : https://github.com/XAI-ANITI/ethik
Alternatively, you can do it yourself it both keras and pytorch, for example by registering a hook in pytorch :
def get_feat_vector(self, img, model):
        with torch.no_grad():
            my_output = None
            
            def my_hook(module_, input_, output_):
                nonlocal my_output
                my_output = output_
            
            a_hook = model.layers[0].register_forward_hook(my_hook)        
            model(img)
            a_hook.remove()
            return my_output

...

for element in val_dataloader:
    model.eval()
    feature_vect = get_feat_vector(element[0].float().cuda(), model)

