Push Button with Voltage Rating (AC) : 125V, Only having power supply of 3.5 or 5V
Your question is wisely asked. In general, an AC-rated switch should be derated for DC use. AC ratings are usually higher than the DC rating, if available, for the exact same switch. There's a reason for that.

AC has zeroes, twice per AC cycle. A manually opened (or closed) switch happens "randomly," relative to the AC cycle. Any "arcing" drawn out as the metal contacts open up will soon be extinguished because the AC cycle will hit one of those zeros.

DC doesn't have any zeros. So there is no inherent self-extinguishing process and therefore they may have to cope with more prolonged arcing when operating at the same voltage.

In your case, the DC voltage is so low and is also so much lower than the AC switch rating, that there is almost certainly no problem using it. I wouldn't worry at all. But it was still wise to ask. (There is also the problem of current compliance, but you've not mentioned it. I'm assuming from the context of your question that we aren't talking about large currents.)

P.S. A manual switch is just a bit of mechanical components arranged to make it easy for a human to operate, and includes metal conductive contacts that are placed into direct contact with each other when the switch is engaged. The metal contacts are just physical bits of metal and do not have to have a specific voltage in order to work. They will make contact, regardless, because the mechanical design for operation is independent of the voltage it is switching.