Bump map appears pixellized
My problem is that my picture is not 32 bit Float contrary to what I thought, and probably also low quality. That said it looks like since 2.8 there's a problem of pixellisation of bump maps in Eevee (link provided by Lemon).
You can smooth the bump by switching the Image Texture from Linear to Cubic.
Another trick to smooth the bump is to give the material a bit of Subsurface Scattering as explained here by Melting Prod (in french). Keep the Radius value low:

Before and after: