Why running Time Machine on OS X Server slows down my Mac?
I found an article that suggests some troubleshooting steps, I'll summarize it here:


Test the drive: copy some files to find out if your read / write speeds are normal. This will help you rule out your internal or server drive.
Check the drive for errors: Again, you can check internal and server drives for errors (bad blocks, permission problems).
Run a general maintenance routine: you can follow the steps they propose here.
Back up in Safe Mode: reducing the system to a bare minimum may help in isolating the problem.
Uninstall or disable drive-monitoring or scanning utilities: Antivirus and other drive scanning software may get triggered by Time Machine activity.
Disable Spotlight: and check for other processes that may be scanning the disk permanently.


There are a couple of additional suggestions on the article, check it out if this steps do not solve your issue and for a full description of each step.

Also, your colleague may provide a good comparison for troubleshooting: you said your hardware is identical, so:


Is the software setup the same? Any differences may point out the culprit.
If software setup is similar, testing hardware on both machines can give you comparison points. For instance, what happens if they use the same Time Machine destination?


Finally, check the Activity Monitor prior to and while Time Machine is running, to single out resource hogs.