image recognition: fully connected network vs CNN
You might be able to get pretty good results on a simple task, but the fact of the matter is that taking random pixels (or indeed just flattening out ALL pixels) essentially destroys any structural information that was contained within the original image.

This was the insight behind convolution networks (from original author Yann LeCun), as they really find areas of correlation in the position/structure of the image input across the dataset. So they understand, for example, the a "1" offers high correlations of pixels in a vertical line, normally in the center of the image input.

This information is no longer included with randomly selected pixels and has been almost destroyed by flattening all pixels into a single vector.



If your use-case requires a certain accuracy and you are reaching that with your simple neural network (or otherwise), then of course it is perfectly valid and you can be happy  :-)