   GPU precision generating planetary normal maps
  I recently moved the normal map generation for my planet renderer onto the GPU. This is fast, but I am running up against what I can only speculate are precision problems with GPU floating point calculations.First the background information: the planet is Earth-dimensions, and the following screenshots are taken a few hundred metres from the surface. Vertices and height maps are calculated in double-precision CPU side, and rounded down to single-precision for upload into floating point textures.From the wireframe, you can see that my noise function is giving plenty of precision:However, when you look at the normal mapped terrain, we seem to be out of precision:At first I wondered whether it was a problem with my normal-map generation algorithm, so I replaced my original, home-grown algorithm with the (considerably more clever) algorithm here, ported to GLSL.Do any of you have any suggestions re dealing with this? Is there some way I can use a pair of floating point buffers to increase the precision in the calculations (similar to simulating a 64-bit integer with 2 32-bit integers)?I think Ysaneya and a few others may have worked around this issue already, given that I don't see this problem in anybody else's blogs/screenshots.   
  The only reasonable way to have the required precision is to get rid of large static coordinate components. If one is 6 million meters away from the center, with 23 bits mantissa it has granularity roughly 0.3m what means it's usable to when vertices are spaced by tens of meters max (and you may want to support older ATI cards with 16bit mantissa ...)So preparing them in doubles on CPU but then rounding to floats before uploading doesn't work either.What you can do is that from a certain LOD level the geometry gets uploaded in tile-local space (the tile center becomes the reference point) and the view matrix is translated too. This removes the static coordinate components (for the specific tile and all its children) and everything works again down to millimeters.   ;Quote:Original post by cameniWhat you can do is that from a certain LOD level the geometry gets uploaded in tile-local space (the tile center becomes the reference point) and the view matrix is translated too. This removes the static coordinate components (for the specific tile and all its children) and everything works again down to millimeters.Sorry, I probably should have specified that I do this already. Tile vertices are calculated in double-precision global coordinates, and then translated into tile-local coordinates, and rounded down to single-precision for upload to OpenGL. As you can see in the wireframe, no issues with precision there.The normal maps are another matter, however. I build a height map on the CPU (normalised to the range [0, 1]), and uploaded as a floating point texture. Then I run a shader to produce a normal map from the height map (using a simplified sobel filter), and rotate the normals on the unit sphere to account for planetary curvature.The normals are nice and smooth at low details, but once you get within about 1 km of the surface, the plateaux shown in the screenshot appear. My terrain features are to a 10x scale with Earth, so the highest mountains are 80 km. 80 km would seem to need about 16 bits to reach 1 m precision, but it doesn't seem to be achieving this...   ;  By curiosity, what resolution is the heightmap for your normal maps ? I gave up on trying to generate it on the CPU since it was very slow.. 30 octaves of noise (combined from many function basis) multiplied by 256^2 or 512^2 and I got hundreds of milliseconds just to generate one tile.Back on topic, to me it looks like a texture precision problem, as if the heights were stored in 8 bits. However you mention that you upload it as a floating point texture. Are you 100% sure that it's not 8 bits ?If you're using fp16, keep in mind that the range stored in fp16 is bigger than [0-1], so you won't get 16 bits of precision in that range. I wouldn't be surprised if you couldn't get more than a few meters of vertical precision that way.The easiest way to test this is to change the heightmap format to fp32. Most likely the problem will disappear. The heightmap is probably just an intermediate texture, so it shouldn't matter if it wastes twice the memory..Y.;Quote:Original post by YsaneyaBy curiosity, what resolution is the heightmap for your normal maps ? I gave up on trying to generate it on the CPU since it was very slow.. 30 octaves of noise (combined from many function basis) multiplied by 256^2 or 512^2 and I got hundreds of milliseconds just to generate one tile.My tiles are currently only 33^2, so CPU side works fine (32 octaves of simplex noise, currently).If you don't mind my asking, how do you generate the heightmaps? Matching CPU side noise functions with GPU sounds like a nightmare - or do you use a small CPU-side heightmap and add extra noise on the GPU?Quote:Back on topic, to me it looks like a texture precision problem, as if the heights were stored in 8 bits. However you mention that you upload it as a floating point texture. Are you 100% sure that it's not 8 bits ?This does seem to be the problem - further testing shows that my little X3100 always returns an 8-bit texture (no matter what internal format I request). I knew Apple's implementation lacked floating-point render buffers, but I hadn't realised it extended to textures as well. I will verify if it works on my PC when I get home.If this does prove the case, I may try encoding an integer height value in 2x 8-bit channels - not the prettiest solution, but may work even on such an anaemic card.   ;Quote:Original post by swiftcoderMy tiles are currently only 33^2, so CPU side works fine (32 octaves of simplex noise, currently).Nice.. but that's not really normal maps if it's not higher res than the geometry, is it ? Are you really storing the normals in a texture ? If so, why not just use the normal attribute per vertex ?Quote:Original post by swiftcoderIf you don't mind my asking, how do you generate the heightmaps? Matching CPU side noise functions with GPU sounds like a nightmare - or do you use a small CPU-side heightmap and add extra noise on the GPU?I went for the nightmare solution :)Y.;Quote:Original post by YsaneyaQuote:Original post by swiftcoderMy tiles are currently only 33^2, so CPU side works fine (32 octaves of simplex noise, currently).Nice.. but that's not really normal maps if it's not higher res than the geometry, is it ? Are you really storing the normals in a texture ? If so, why not just use the normal attribute per vertex?Sorry, was tired when I wrote that. The tiles are 33^2 vertices, and have a 128^2 heightmap. This is small enough to be generated in one frame, but unfortunately large enough to cause a frame-rate spike each time a tile loads.Quote:Quote:Original post by swiftcoderIf you don't mind my asking, how do you generate the heightmaps? Matching CPU side noise functions with GPU sounds like a nightmare - or do you use a small CPU-side heightmap and add extra noise on the GPU?I went for the nightmare solution :)I think I will explore that in the near future. I have to come up with a decent shader combination system first though, as I don't want to hardcode the whole procedural setup.The other alternative is to multi-thread the tile loader, but I have some doubts that I will gain enough performance to allow high-resolution normal maps, and the code complexity goes through the roof.   ;  I'm curious if you got the 'considerably more clever' (tm) normal map generation to work right?  In my next post I had to admit that the normal maps did not line up at the cube face edges (sadly). Maybe this is not such an issue on planets of huge scale?My current implementation still generates the normals in object space doing the traditional cross product normal with normal smoothing algorithm, and it is on the CPU.Also, I'm using pthreads to generate heightfields - I spawn 4 threads that generate heightmaps whenever there are any needed.  I had to set the thread priority really low in order for the threads to help the framerate/smoothness any.  If I used threads and didn't mess with the priorities it did me no good.32 octaves noise?!  My tiles are only using 4 octaves simplex noise *cry* - of course I'm going for a much less realistic look.I'm really curious how Ysaneya generates his normals :DBtw Ysaneya, I finally got a texture implementation working similar to your new method (blending 16 textures using look up table) - thanks for sharing!   ;  yes definitly looks like all the heights are stored in 8bit (ive seen this before)I think u mentioned u canA/ store in 2 channels + unpack float height = tex.r*256 + tex.gB/ instead of storing heights store the offset from the neightbouring pixel (similar to normalmap), harder than u think to do though, A is a better option unless youre trying to save memory   ;  So I moved development over to my PC with an ATI 4870, and switched the height and normal maps over to use floating point textures, with the result that most of my precision problems are gone.Unfortunately, I still have precision problems when I reach the ground. The following shot is from an altitude of 2 metres:I think the problems are due to the sobel-filter I am using for normal map generation, given the squareness of the artefacts, but they only occur at these very low altitudes.Quote:Original post by petrocketI'm curious if you got the 'considerably more clever' (tm) normal map generation to work right?  In my next post I had to admit that the normal maps did not line up at the cube face edges (sadly).Seems to work perfectly - although I haven't quite sorted the precision problems.Quote:Also, I'm using pthreads to generate heightfields - I spawn 4 threads that generate heightmaps whenever there are any needed.  I had to set the thread priority really low in order for the threads to help the framerate/smoothness any.  If I used threads and didn't mess with the priorities it did me no good.32 octaves noise?!  My tiles are only using 4 octaves simplex noise *cry* - of course I'm going for a much less realistic look.Well, if you want to reach the ground on an Earth-scale planet, you need 16-24 octaves of noise (depending on algorithm). At this point, the CPU-side generation is much too slow, so I followed Ysaneya's suggestion, and ported all the noise generation over to the GPU.Amusingly, it takes less time to generate a 512x512 heightmap and normal map on the GPU, than it does to generate a 17x17 vertex patch on the CPU...  In general, I can generate about 4 complete tiles per frame without badly impacting framerates.   