Updating sub panel in shed
You don't need to upgrade the supply breaker. 

The sub-panel can be any size 50 amps or up. Sub-panel "size" is more about space than amperage, at any point past "at least as big as the supply" and space is a good thing to have which normally does not cost much.

It's often worth comparing the price of basic "main" panels that move in volume (100A, 200A) to the cost of a sub-panel which are sometimes MORE expensive, because they sell a lot less of them, or they cost a trivial amount less.

It's perfectly fine to feed a 100A(or whatever) main breaker from a 50 A breaker - the 100A breaker will be serving as a disconnect switch, and you will remove the bonding screw in that panel so it IS a sub-panel. Then you'll have plenty of space for anything. 

In most cases the best way to add a wire in existing conduit (unless someone left you a pull string, and sometimes even then) is to pull out the wires in it now (and use them to pull in a pulling rope as you do), add the new wire, and pull all the wires back in together. Green insulated may be a better choice than bare for your grounding wire; Among other considerations, stranded pulls a lot easier than solid.

In the new sub-panel with a 4-wire feed, you'll keep the grounds and neutrals isolated (removal of the bond screw and using a separate bar for grounding connections) and you will either tie the grounds into the grounding electrode system the shed (as a separate structure) should already have had, or add that and bond it to your ground as well.