Enterprise Mac Mini Server - Remote Desktop
You get two graphical interface log in sessions simultaneously on OS X (server.app or without server.app) and there is no configuration needed as this is the default settings. Just use any VNC client and set up two or more users local to the server.

You'll want to set up the Mac Mini with Xcode Server running and then have each developer run Xcode locally. When they make code changes, you can have the server compile things "continuous integration" style.

Similarly, you could just run your own CI infrastructure on the Mini running whatever OS you wish, but count on the server to only let two workers (either one local and one remote or both remote) to run Xcode GUI itself from each Mac.