Use SHAP Values for PyTorch RNN / LSTM
Is there a way to do the above?
The SHAP Package is very helpful and works pretty well for PyTorch Neural Nets. For PyTorch RNNs i get the error message below (for LSTMs its the same):



Seems like it doesn't work but is there a workaround or something? Does anyone have experience with PyTorch and SHAP?

'RNNs aren't yet supported for the PyTorch DeepExplainer (A warning pops up to let you know which modules aren't supported yet: Warning: unrecognized nn.Module: RNN). In this case, the explainer assumes the module is linear, and makes no change to the gradient. Since RNNs contain nonlinearities, this is probably contributing to the problem.' That was an answer I found at Shap. 

Try to check captum.ai that is built on PyTorch.
