Data structures that store efficiently detected features of an image I struggle to find this, suppose I apply any feature detection algorithm. From my references it seems to me it's not usually explained what kind of data structures would be suitable for storing such features.

For example in opencv it seems to me that the Canny Edge detector will return a matrix (I assume the code/gray level code is like relating high pixel value with the the feature in that specific pixel). I think it would be useful to know what data structures could actually be useful in this case to navigate through all the extracted features instead of iterating through all the pixels.

So the question is... in general (or at least line features) what data structure is usually useful? and how is such data structure constructed?

As a specific example... I'd like to extract all the edges of a given image, name such edges $e_1,\ldots,e_n$. Then I need to access to the pixels that lie both to the left and right of such edge(not all, just pixels that are close enough to $e_i$, for all $i$). Finally I need to perform some operations on such pixels.

To summarize something like:

$e_1,\ldots,e_n \leftarrow extractEdges(I)$

$\forall i \in \left\{ 1,\ldots, n\right\}$

$\;\; S_l \leftarrow pixelsLeft(e_i)$

$\;\; S_r \leftarrow pixelsRight(e_i)$

$\; \; doSomething(S_l,S_r)$

Above $I$ representes the input image.

Thank you
A simple solution would be a 2D array containing a 1 in the locations that are detected as an edge, or a sparse 2D matrix.  A more sophisticated solution would be a quadtree.