Given a 12x12 binary image (only black and white pixels) what is its dimensionality? And how can I define dimensionality of a data space?
There is some ambiguity in dimensionality you ask for. The vector space that your input lies in is $12\times12 = 144$ dimensional. So, if you're going to apply some kind of dimensionality reduction algorithm, e.g. PCA, this is the dimension. But, we sometimes tend to refer to dimension as the shape of the tensor, which is $2$ in this case. Some libraries might read the image into $12\times12\times3$ tensors, in which the third entry indicates RGB decomposition, even if the image is black and white. In this case, the tensor dimension is $3$.