What info should I look for when choosing a solid state relay for arduino?
If you're only controlling a button, the load current and load voltage are pretty much irrelevant since I'd be surprised if you even used 5% of it's rating in either category. I can't tell you how much you'll need here since I don't know the voltage that the relay will be operated on and the resistance of the path for the load.

What you do need to look at is the voltage and current of the input/control/logic signal. It should allow a 5V control voltage and the current draw for that logic signal should be no more than 40mA, although it's ideal to stick to a lower number (say, 30mA).

However, manually picking out a relay manually isn't ideal for a beginner since it requires a bit of extra components to properly run. A relay module (Amazon) allows you just hook up your wires to the jumpers and terminals of the relay module without any extra soldering (as far as I know they aren't breadboard compatible, thus requiring a custom PCB or having it "flop") or parts.