Potentiometer with switch wiring question. Not reading correctly I have a potentiometer with switch, that seems to be working a little wonky (Keep in mind I have tried this with a few different pots and they all behave the same way which leads me to believe I am wiring them wrong).

I can easily hookup the potentiometer and it's three terminals on the side, 1 being ground, 2 being output, 3 being input power. I can monitor it's value on a serial port with Arduino.  However when I try and test the switch, which are two terminals on the bottom of the pot.  I get an on signal when the pot is on, but when it is clicked off I get a random on/off signal.  As in it reads that it is sort of on but never truly reads off.

Right now I am simply giving it 5v from the Arduino to both the switch and the third terminal.  Should I be giving the switch power via a capacitor? Not sure, have not tried that yet but that may smooth out my readings.

Any suggestions?
From your description of switch wiring, it sounds like you have one of the switch leads attached to +5 V and the other switch lead to an input.
As noted in comments and answers to  “Why all unset pins of my Arduino Uno are outputting 2 volts?”, such an input will read high when the switch is closed, and will read randomly when it is open, because an input is a high-impedance device that can accumulate charge:

... input impedance typically can be taken as 100 MΩ for design purposes. ... Each pin has a few pF (5-10?) of capacitance ... which in general is small compared to stray capacitances of attached wiring.
When stray electromagnetic fields charge the pin and stray capacitances, a small voltage develops on the pin. You could connect a 1 to 10 MΩ resistor between the pin and ground, to avoid the [voltage] you are seeing.

The easy fix for the problem is to attach one switch-lead to ground, one to an input pin, and initialize the pin with INPUT_PULLUP pinmode instead of just INPUT.