Distorted normal map on smooth surface Blender rookie here. I'm learning about baking normal maps in 2.80 with some pretty basic setup.

I'm getting strange results on a smooth metallic flat surface. You can see it on the following screenshot. I'd expect the surface to be completely smooth, but it looks sort of like alligator skin or something :). Any idea what can cause it?



Things I did (roughly):


Created a low-poly cube and UV unwrapped it (for this basic example I assumed "Smart UV project" is good enough) onto a 4K image (seems like overkill but I thought the problem originates from too low resolution - it doesn't seem to be the case though).
Created a high-poly copy with Bevel, Subsurf and a slight Displacement modifiers, which I then applied.
Set Smooth Shading on both objects.
Added material to low-poly cube (metallic) with Texture Image node (Color space: Non-Color).
Baked normals from hi-poly to low-poly.
Saved image, selected it in Texture Image node and connected the node to Normal Map and then to Principled BSDF's normal input.


Here's how the meshes look:



Normal map image:


Things I tried:
Adding more detail (loop cuts) to the hi-poly doesn't fix it. Setting resolution of normal image to 4K didn't help either.

Here's the complete .blend file with the normal map image: cube.zip

Thanks in advance.

Update

After attempting to UV unwrap the low-poly mesh according to hatinacat2000's answer below, I ended up with following normal map:



However, baking that in did not remove the weird effect. To be sure, I mean that the metallic cube looks like there's some transparent net spread over its sides, not the displacement which I applied intentionally on the hi-poly mesh, and it seems to work fine.
The problem is numerical error. 8 bits can't resolve the direction in  a surface so sensitive to minute changes in angle.




Set up a new 32-bit image.. you can do this in the Image Texture node itself
Set its color-space to Non-Color
Bake as before.


If you want to save the image out, 16-bit PNG will probably do.