Corrosion of a galvanic couple made with silver and gold
As you state, though silver is not very reactive, silver jewelry would corrode more rapidly when in contact with gold and a bit of sweat as electrolyte. Though the difference in electronegativity of pure gold and pure silver is 0.15V, and is considered acceptably corrosion resistant, commercial jewelry contains copper and other metals in the alloys. There is clearly evident corrosion in photos of copper/silver jewelry. According to the Victoria and Albert Museum staff, "All metals, with the exception of 24 carat gold, suffer the effects of corrosion."

However, in a bangle, physical abrasion is likely to cause more damage and loss of material than galvanic corrosion. Wear the jewelry and enjoy it, but it will not last forever.