How do I properly size the ground wire for a subpanel?
YES you can use #4 Al as your ground.  Your 1AWG Al is good for a 100 AMP feeder (actually 110).  From NEC 250.122 this requires an #8 CU or #6 Al ground wire as a minimum (or 6CU and 4 Al if you are really running a 110Amp Feeder).
As a reminder, your feeder will be 4 wires, the two hots and the neutral will be #1AL (unless you've de-rated the neutral to 70% or from specific load balancing calculations if that is allowed in your area).

NEC 250.122 table: