What is the point of tensors in CNNs? Why not simply reshape the data into matrices?
Tensors come pretty natural in convolutionals networks.


Local pixel information matters: if $e$ is a pixel in your example above, it's important to know that $a$ through $i$ are its neighbors. This information gets lost when you reshape an image into a vector. Look how a convolutional layer works.





Training usually is done in batches, which is another dimension as far as a neural network is concerned. You don't want to mix different images into a matrix, they are totally independent.
Finally, there is depth channel. Initially it's one of R, G, B channels of the input image, and then each channel corresponds to a filter applied to the previous conv layer. Once again, filters are independent, it doesn't make sense to mix them up until the final layer.


So, in total there are 4-rank tensors going through the conv net. It's not only more intuitive (each dimension has a meaning), but also results in higher accuracy, because it employs all meaningful information from the images.