Identifying “clusters” or “groups” in a matrix
What do you think is distance measure in your case?

I assume there are three dimensions here:


RowN (row number)
ColN (column number)
Value (value: A, B or C)


That means data you get from 4x5 matrix looks like:

Sample1 -&gt; (1, 1, A)
Sample2 -&gt; (1, 2, B)
...
Sample5 -&gt; (1, 5, A)
Sample6 -&gt; (2, 1, A)
...
Sample15 -&gt; (3, 5, C)
...
Sample20 -&gt; (4, 5, A)


Is value scaled? In other words, is A &lt; B &lt; C? 

If yes, then


you can replace {A, B, C} with {0, 1, 2} (or may be {10, 11, 12}, if you want this difference be less important than RowN and ColN attributes)
normalize your data
use, for example, K-Means clustering algorithm (http://stat.ethz.ch/R-manual/R-patched/library/stats/html/kmeans.html) from stats R package


In that case the distance between two will be:

Sqrt( (RowN1-RowN2)^2 + (ColN1-ColN2)^2 + (Value1-Value2)^2 )


If value is not scaled (regular categorical variable), use some modifications of K-Means that work with categorical data.

So in case of 100x100 matrix you have 10000 observations and three variables, which is pretty trivial sample size.