Is there a maximum number of remote connections that can be setup on mac mini?
Not sure if it will help, but I would update 10.9.4 to 10.9.5 as I found 10.9.4 to have a few issues on the servers and Mac I service for clients. 

Note: I can't think of any compatibility issues that would be an issues with that update, but assess the software used before you update of course.