Pulldown Resistor on a Unity Gain Buffer
Adding a pullup or pulldown between the Arduino and the buffer won't do you any good (assuming the buffer is actually doing its job).

I assume your buffer has a CMOS input stage.  That means its input resistance is so high that miniscule currents, especially when exacerbated by dirty power, cause voltage swings and oscillations.  This is where to put your pullup or pulldown, on the buffer's input.  Even a huge resistor (hundreds of K or even megs) should work.

If the signal you're measuring is so weak that no pullup or pulldown can be used (say, the output from a load cell), you'll need to find a more stable amplifier, possibly one with a bipolar input stage.

If you're interested, read up on op amps and input resistance.