Retina display support for XQuartz X11 applications
Yes, you are correct. The problem is X11, XQuartz and their lack of support for the Retina Display. This is a long known issue.

In the linked thread, jeremyhu states:


  X11 just doesn't work well for resolutions outside of the 72-90 DPI range. This is an upstream X11 issue that needs to be solved before we can really do anything about it in XQuartz.




Not sure this will help you, but at least since last August, RealVNC has supported Retina displays.



Also you might try running X11 with your Retinia display at full resolution to see if this mitigates the issue. You can accomplish this with SwitchResX, among other software that is available to access other resolutions.

SwitchResX is a commercial third party application, but it is available to try for free and for download. Probably because it is technically a hack to use the full resolution of Retina Displays, it appears to be unavailable on the Mac App Store.



Another option to try is using NoMachine's nxclient/nxserver technology. I can't find anything that says it has Retina Display support, but a post in their forums leads me to believe that there is at least some support for Retina Display resolutions in the current version.