Amp size for a sub panel to a shed?
I would take a very close look at that interior panel.   From the look of your outside 4-space "panel", it looks to be your main service panel, and contains your main breaker and your neutral-ground bond.  It's a bit like a "rule of six" panel: it appears you can/must shut off both breakers independently, e.g. The A/C will still have power if you shut off the 100A. 

As such, the interior panel is a subpanel.  It doesn't need a main breaker since the main panel is attached to the building.  

So I would take a very, very close look at the panel interior and look at Siemens' line of panels.  What I'd be looking for is a 16-20 space subpanel that is the exact same physical form-factor as your panel internally. That is to say, all the bolt holes are in the same place, and you could unbolt the guts of the new panel and they would just bolt up  into your existing panel chassis. 

Because the hairy-scary thing about changing service panels isn't the guts, it's uncabling/unplumbing all the cable connections around the edges of the panel.  It's fairly easy to pop the breakers off (you don't even need to remove the wires) and remove the feeders from their lugs.  Tuck it all back so you can get to the main busbar assembly, then swap it with one with more spaces.  

From there, I would move the A/C load into the RV's main panel.  Then use the breaker formerly for A/C, and use that to feed the shop subpanel.  

The main panel's breakers total 125A, a common electrical service.  I would not allow them to total more unless you consult with the electric company about the service they are providing you, and whether it can handle more.