How to calculate KL-divergence between matrices
I think you can. Just normalize both of the vectors to be sure they are distributions. Then you can apply the kl divergence .
Note the following:
- you need to use a very small value when calculating the kl-d to avoid division by zero. In other words , replace any zero value with ver small value 
- kl-d is not a metric . Kl(AB) does not equal KL(BA) . If you are interested in it as a metric you have to use the symmetric kl = (Kl(AB) +KL(BA) )/2