Stainless steel bolts for brake calipers
The braking loads are unlikely to be enough to exceed the tensile strength of a 6 mm bolt – forgive my butchering of units here, but it would take something like 2,250 pounds of force to make a 6 mm bolt fail (75,000 psi tensile strength for type 18-8 or 316 stainless and about 0.03 square inches in the bolt works out to around 2,250 pounds of force). That's load at failure, but it seems like the limiting load on the brakes will be about the weight of the bike and rider. That means that even a big rider on a heavy touring bike isn't capable of much more than about 400 pounds of load. I found my data here.

So structurally, I think you're sound. You can have a pretty bike.