
Outline of this article

Deep Dive into matrix storage/memory representation in C++. Introduction to Row and Column major order.
Impact of looping over a matrix in the same or different order as the storage representation, along with an example.
Introduction to Cachegrind; a tool to inspect the cache friendliness of your code.
Memory formats supported by PyTorch Operators.
Best practices example to ensure efficient model execution with XNNPACK optimizations

Matrix Storage Representation in C++
Images are fed into PyTorch ML models as multi-dimensional Tensors. These Tensors have specific memory formats. To understand this concept better, let’s take a look at how a 2-d matrix may be stored in memory.
Broadly speaking, there are 2 main ways of efficiently storing multi-dimensional data in memory.