Scattering parameters of a phase shifter
I'm no waveguide guy, but I've spent a bit of time fussing with VNAs.

The information I found about the X885A phase shifter was from the January 1955(!) Hewlett-Packard Journal. 

First, it appears the X885A phase shifter is designed for operation between 8.2 and 12.4 GHz (excuse me, 8.2 to 12.4 kilo-megacycles). You should not expect it to be well matched outside this band. 

Second, the typical VSWR is given as about 1.2 in the operating band. This means an \$S_{11}\$ or \$S_{22}\$ magnitude of about 0.1.

Third, your phase plot looks like what happens when no signal is received by the VNA and you're basically looking at the phase of the receiver noise. 

My advice is to re-measure the device and be sure to record the S-parameter magnitudes as well as the phase, so you can know whether you are actually measuring a valid signal. 

I don't know how VNA measurements on waveguide work, but it may be that the feed  from the VNA output to the waveguide is simply not very efficient, or the actual VSWR is much better than the 1.2 spec, so that the reflection is not enough to achieve a low-noise measurement. Under this theory, the reason you're seeing a reasonable phase measurement below 7 GHz is because of very high \$S_{11}\$ in this frequency range.