Given matrix, count paths visiting each number exactly once
If I understand well, you are looking for an Hamiltonian path which is basically NP-complete. With your DP idea, there is $2^{21} = 2M$ possible bitmasks and there is no way to order them. So I am not sure to understand how it works.

In fact you can start building a graph and get rid of any direction/"-1 cells" consideration. In your example you would get a simple chain from 1 to 21.

Then, in the very general case you may have to explore all possible paths (using DFS for instance). But in this kind of problem you can look for some tricks. The point is to select the 20 edges of the path:


Any vertex having only one edge is either start/end (NB a reversed valid path is valid like in your example) and its edge is selected
A vertex having 2 edges have these edges selected (unless it is start or end)
A vertex having 2 edges selected, have all the other edges removed.