Is there a way to store matrices in an FPGA?
Your 'most limited resource' is likely going to be the number of multipliers on the FPGA. 

Your matrix multiplication logic will then consists of a stream of numbers read from a source, fed into a bundle of multipliers and the result stored again.

Two-dimensional access is easiest implemented using register albeit it also uses a lot of logic to select the outputs. I have implemented a two-dimensional memory using standard (one dimensional) memory chips but that was patented. I think you will just need to spend some cycle 'rotating' the matrix. Or be clever how you write the result. You might even consider storing each matrix twice. 

I am more concerned about precision.
I have no idea what your numbers are, but I do recollect colleges at Delft University about the problems with maintaining precision when repeatedly using floating point numbers. In your case it will be worse because the multiplier in an FPGA has only a limited number of integer bits. Unless you want to implement massive parallel floating point multiplication.