Reflectometer - directional coupler
Doing a few calculations it seems to me that your measurements are correct. Let's take for example:

$$ 100R//10 pF = (@ 868 MHz) = 3.25-17.7j $$
Then if we calculate the reflection coefficient:
$$ \rho=\frac {Z_L-Z_0} {Z_L+Z_0} =-0.69 -0.56 j$$
The phase of \$\rho\$ is -141º=219º. In comparison the phase of \$\rho\$ for a 100R load is 0º.

In your measurements with 100R//10pF you have 0.58 V -> \$\pm 58º\$ and with 100R you have 1.455 V -> \$\pm 145.5º\$. The \$\pm \$ accounts for the phase sign ambiguity of the used detector. Therefore the absolute phase difference is either \$145.5-58º=87º\$ or \$145.5+58º=203º\$, (we can't know which one to take because of the measurement phase ambiguity). We note that \$203º\$ and \$219º\$ are close enough.

If we repeat for 100R//3.3 pF we have that the phase of \$\rho\$ is \$-92º=268º\$. According to your measurements the phase difference between 100R//3.3pF and 100R is either \$145.5-118=27.5º\$ or \$145.5+118=263.5º\$ and \$263.5º\$ is quite close to \$268º\$.