Better cap needed for toilet bolts
A simple fix might be using white poster putty / sticky tack.  If that doesn't hold, try a dab of superglue.  Each of these are available at the dollar store.  For a slightly more expensive option, perhaps some white adhesive silicone caulking.  All of these options should hold the cap in place enough to not blow off in the wind, but stilll allow you to break open the cap again if you need to ever remove or tighten the bolts.