Can Frameless rendering reduce latency? And, can FPGA do 3D rendering instead of GPU? In virtual reality, the motion-to-photon time is very important. Oculus says it has to be less than 20ms. Maybe 10~15ms is better.

Some people try to introduce Frameless rendering technology into VR. I think it's good to reduce the latency. But I don't know how the display quality is. According to the paper: "Adaptive Frameless Rendering" and the video (https://www.youtube.com/watch?v=ycSpSSt-yVs), it seems not very good. But, in this paper "Construction and Evaluation of an Ultra Low Latency Frameless Renderer for VR", the authors used FPGA instead of GPU to render and used Frameless Rendering to reduce the latency to 1ms. 

In my mind, FPGA is very hard to replace GPU in 3D rendering area. And, Frameless Rendering needs "racing the beam", it is also very hard to program. 

What do you think? Is it realistic to use FPGA and frameless rendering?
First of all, the "frameless rendering" technique is in the context of raytracing, not rasterization. It's not obvious how it could be made to work effectively with rasterization, given that the basic idea of it is to update an image by a combination of temporal reprojection plus firing rays specifically at areas where the algorithm thinks the image is undersampled.

So this technique is, prima facie, not compatible with rasterization-based graphics applications. But if you're already doing raytracing for other reasons, this technique would be interesting to look at; it certainly appears to improve quality relative to an image raytraced from scratch each frame, with the same number of rays per second.

Raytracing on a GPU is certainly possible; you don't need an FPGA for that. I've only skimmed the second paper, but my reading of it is that the main reason for the FPGA is to get a close coupling between the display scanout and the rendering activity: namely they "race the beam" and evaluate pixels just before they're about to be scanned out, thus obtaining low latency.

The GPU equivalent of this is probably to split the image in thin horizontal strips, and kick off a compute dispatch to render each strip just before it starts to be scanned out. Today, this is difficult to accomplish as it requires either millisecond-precise scheduling that desktop OSes are not currently set up for, or it requires the GPU to be able to dispatch based on an interrupt from the scanout unit&mdash;a hardware feature that doesn't currently exist AFAIK (or if it does, it isn't exposed in any graphics APIs). Or you might be able to make it work with a long-running asynchronous compute dispatch, if you can find a way to stay in sync with scanout.

So, there are obstacles, but they aren't insurmountable, and I think if there was sufficient interest in racing-the-beam-style rendering, then OS and GPU vendors could come up with a way to do it in the future. So I don't think an FPGA is required for this kind of technique to work. On the other hand, the fact that it's based on raytracing is a much bigger obstacle to using it in "real-world" apps and games.