How to draw surface normals from surface normal maps? Using deep learning to estimate surface normals from monocular RGB images is a common task. The resulting image generally looks like this:

My question is how can I use this map to actually draw a normal vector (arrow) for any given point on the image? For example, I want to draw a line on the countertop that points up.
Most relevant tidbit I found on the topic: &quot;The RGB color channels (red, green, and blue) in a normal map correspond to the respective X, Y, and Z coordinates of surface normals.&quot;
But I don't get what this means. The values are standard RGB, i.e., for each coordinate, you get three numbers in range [0, 255]. Just by how color changes between surfaces, it is clearly related to normals (obviously) but when I tried quiver plots using matlab I couldn't get reasonable results.
TLDR: What is the relation between surface normal maps and actual normal vectors? How can I draw a 3d line given a point on the image which represents the normal vector perpendicular to that surface at that point?
Typically the normal vector, which has XYZ components in [−1, 1] is linearly mapped to the RGB range [0, 255]. You can retrieve the normal vector by reversing this mapping.
For completeness, the encoding is:
$$
r = \text{round}((0.5x + 0.5) \cdot 255) \\
g = \text{round}((0.5y + 0.5) \cdot 255) \\
b = \text{round}((0.5z + 0.5) \cdot 255)
$$
Decoding is:
$$
x = (r/255) \cdot 2 - 1 \\
y = (g/255) \cdot 2 - 1 \\
z = (b/255) \cdot 2 - 1
$$