High side switch and Low side switch
I wonder what is the difference between high side switch and low side
  switch. Basically it work the same way (electrically speaking) but I
  don't know if one is better than the other.


It depends on the circuit you are switching. Ignoring safety in vehicle reasons, you might choose to switch on the high side of a signal amplifier because the low side might be the signal 0 V reference (as well as power 0 V) and, routing both signal and power grounds through a common point i.e. the low side switching MOSFET can lead to serious noise and distortion effects on the final signal. If the MOSFET had true micro-ohm (or less) on-resistance then it's less of a problem but you won't find one this good for pennies or pounds.

So, if you can switch the low-side it's generally easier because a micro/controller will normally be logic level referenced to 0 V and it can easily drive an N channel MOSFET (or NPN) but beware of signal lines sharing this common switching device as outlined above.


  What is the best solution if there is big capacitor in the circuit?


If you are referring to a big capacitor across the target load's power pins then there are no real implications other than to ensure your switching mechanism can handle the in-rush current.