Is it acceptable to leave a capacitor across batteries even when turned off?
Leaving a capacitor across the battery is fine, but read the datasheet of the capacitor you will use (or find a similar datasheet if you are using random junkbox parts). Ceramic caps will have leakage perhaps in the nA or less, whereas an electrolytic cap might be in the uA or 10's of uA range after settling for a while.

Compare this leakage current against the battery's expected self-discharge rate in its datasheet. Ballpark figure for alkaline batteries is about 80% of original capacity after one year. So let's say you are using AA batteries:


AA battery capacity = 2500 mAh
80% remaining capacity after 1 year


So 20% * 2500 mAh = 500 mAh of capacity removed after 1 year

So the equivalent discharge = 500 mAh / (365 * 24 hours) = 0.057 mA = 57 uA

So maybe you want to keep the leakage current less than say 6 uA. If you're using little lithium batteries, or watch batteries, you would want way less. You can repeat the same calculations for your situation.

As the other poster mentioned, the second circuit is bad because you will have arcing whenever the switch is closed, greatly shortening the life of the switch. Even with a 1 uF cap, if it is low-ESR such as ceramic the inrush current can be 10 Amps or more if you watch it on an oscilloscope.