Multiplexing and electric capacitance
I get the problem. It's not related to the 4066 or anything. It's that good ol breadboard.

I'm working with little capacitance difference and the breadboard itself have a capacitance. One or two connection, well that's ok, But I was having a long bus with at least 30 wires (so times 2 for pins connection) in series to connect the input of the AtMega328 with the 1M\$\Omega\$ to all the output of the 4066. The result was a big disturbance in the force ;)

I've done a test on a board with the chip soldered, barely not problems. Will just have to split in two groups, but not a big deal as I've got plenty of inputs remaining.