Reading a resolver sensor with a long cable
The presence of harmonics suggests simply that the waveform is being distorted. 500 meters each way is a long run, and even more so for 7.5 KHz. Assuming that the excitation is a nice clean, low-distortion sine wave, have you measured the amplitude of the converter inputs? I'd guess you need some sort of receiver amplifier to get the signals up to specified amplitude. If you already have such amplifiers, check to see if they are producing the distortion. 

The obvious tool for this job is a spectrum analyzer. You do not state the harmonic distortion levels required to get your (excellent) error fit. If it's a few per cent, you probably can't see it on an oscilloscope.

So. Levels first. Actually, you need to check the amplitude levels at the resolver first, and adjust the driver amplitude to get the correct level. Then look at the converter and amplify as necessary. And note that if you've adjusted the drive level already to get proper converter level, you may be overdriving the resolver, and producing distortion that way.

You say the lines are shielded. Are they individually shielded or are they a multipair cable with a common shield? And while it's probably not your problem, be aware that in some cases where pickup is a problem, individual shields with an overall shield has been used. 

There is an obvious problem with my advice so far, but I don't know how to get around it. Your cable runs are already in place, and due to radiation concerns you probably can't get to the resolver to measure its levels. In that case, you're just going to have to try blind experimenting with drive levels, monitoring the distortion levels of the returns with a spectrum analyzer.