Xserve(s): should I reinstall OS X
It's a good idea to reformat the machine, because who knows what all was run on it, and whether that may cause problems for you later or more so now. So it's a very good idea starting off from a clean slate - you will know what is installed, since you installed it.

In fact, I would never use a re-purposed machine without formatting it first. It's a security risk not formatting, and I would not want to chance my data etc on it. And since the machine was a server connected to the internet, there's even more reason to reformat it.

It will take you much less time to just start over than it would to go through potentially hundreds of thousands of files located on the system that could have been modified intentionally or maliciously.

A typical install of OS X usually takes less than 1 hour, usually around 35 minutes or so.  Server may take longer, and all you really need to do is get it started which takes less than 5 minutes.

Additionally, you would not even need the original install disks, you can install any version of Mac OS X newer than what came with the server originally. Which might not be a bad idea since you are not really needing the features that a server version has to offer, based on what you said your were doing with the systems.  But that really depends on what the Mathematica setup requires.