ADC is detecting a sine wave as several clusters of points
From looking at the last picture, we see that multiple votlages are mapped to the same discrete value, this means that some bits will not get transmitted/are lost. It seems we always have four repetitions that could be &quot;pulled apart&quot; again to form a nice continuous curve: This tells us that there must be two consecutive bits that are lost. After a little bit of trial and error I noticed that you cannot just zero them out due to the even distribution of those large &quot;steps&quot; across zero: If we instead implant the first two significant bits at the bits that are lost, we get exactly what we are lookign for:
So basically I'd say the DAC outputs are
[d1, d2, d3, d4, d5, d6,...]

(where dn represents the the n-th most significant bit), but your program somehow reads
[d1, d2, d1, d2, d5, d6,...]

Actually it might also be d(n),d(n+1) that get overwritten by d1, d2 for some n, as I don't know what resolution I'm looking at in your plot.
I made a quick plot to simulate this behaviour, and we can see pretty much the exact same outcome:

Here is the MATLAB code used written for this plot:
bits = 9;
x = 0:1e-2:2*pi;
y = round((sin(x)*0.85 + 1.15)/2 * (2^bits-1));
yd = nan(size(y));
perm = [1,2,1,2,5,6,7,8,9];
for i = 1:numel(x);
    bin = dec2bin(y(i), bits);
    bin = bin(perm);
    yd(i) = bin2dec(bin);
end
plot(x, yd,'.-b');