Design approach for a servo and LDO for rest of the circuitry
Now my circuit works fine for most of the time, but sometimes servo
fails to operate the load.

You try the simplest things first and that means changing the servo drive from a feeble MMBT2222 to an n channel MOSFET with low on resistance: -

If that fails to work correctly then you use a boost converter to produce a higher voltage.

I could increase the cap from 100uF to possibly 1F cap

Unless you have a particular problem that is seen at (for example) low temperatures where battery drive strength may be impaired, this doesn't sound like a good idea. Do the MOSFET thing first.