Change scaling algorithm for non-retina apps running in low resolution on a retina display?
I don't have a system on which I can reproduce your issue, so this is just a guess, but check out the advice on this forum post: https://forums.macrumors.com/threads/change-osx-scaling-mode-for-retina-screen-with-non-retina-content.1428540/
Namely:

Try using SwitchResX to enable 1:1 mode at your display's native resolution (I believe this will make your content very small; not sure whether that's a workable solution for you)
Use an external, non-retina display :)