ADC strange behavior (ATmega)
The spikes are clearly clipping to very particular values (probably a power of two) or you are using signed integers that are too small to contain the whole value.

You didn't supply full source code so I'm guessing here. You probably try to fit the measured value into a too small integer. Make a dump of the vaules you get and find the peaks in them. Then look at the binary representation of these numbers and see if you can find something in common.

With quick estimation the distance between two peaks is about 1.28V, a little bit too coincidentially close to the size of a signed 8 bit integer.

I advice to use type definitions like int8_t [-128:127], uint8_t [0:255], int16_t [-32768:32767], uint16_t [0:65535], int32_t [-2147483648:2147483647] and uint32_t [0:4294967295], which clearly shows how large the variable is.