Xcode Service: Do I need a separate Mac for OS X Server on Mavericks?
The short answer is, yes, you can install OS X Server and the Xcode Service on your development iMac. You do not need another Mac for the server.
In detail, there are two parts to this question:
1. Can you install OS X Server on your development Mac?
Yes, you can install OS X Server on your development iMac. You do not need another Mac for the server.
From the time of OS X Lion, OS X Server is just another application you buy from the Mac App Store and install. You can install it on any Mac that already has the OS X client version.
The requirements to run OS X Mavericks Server are:

Mac computer running OS X Mavericks.
2GB of memory.
10GB of available disk space; some features require additional disk space.

2. Can you install the Xcode Service on the development Mac (without using another Mac as a server)?
Yes, you can install the Xcode Service on your development Mac. The Xcode Continuous Integration Guide states (emphasis mine):

Note: A continuous integration workflow typically relies on one or more development Mac computers running Xcode and on a separate server running the Xcode service. However, you can install OS X Server and run the Xcode service on your development Mac. Such a configuration can be helpful for evaluating how to adopt continuous integration. Afterward, you’ll find it more useful to have a dedicated server running the Xcode service, hosting your repositories, and remotely performing integrations.