Improve performance on 4 x 4K mosaic with FFmpeg
The problem was about the overlay filter which is too slow. I've changed for a combination of hstack and vstack as you can see here :

ffmpeg \
-hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
-hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
-hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
-hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
-filter_complex " \
[0:v] scale_npp=1280:720, hwdownload, format=nv12 [upperleft]; \
[1:v] scale_npp=1280:720, hwdownload, format=nv12 [upperright]; \
[2:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerleft]; \
[3:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerright]; \
[upperleft][upperright] hstack [upperline]; \
[lowerleft][lowerright] hstack [lowerline]; \
[upperline][lowerline] vstack [mosaic]; \
[mosaic] hwupload_cuda" \
-c:v h264_nvenc -preset llhp -profile:v baseline -f matroska pipe:1 | ffplay -i -


It works very well and I even can build a 1080p mosaic using lossless preset and high444p profile.