circuit protection during supply plug in
What are the other precautions or design mechanisms I can take to
  protect my circuit from damage ?


Plenty of things you can do - you might consider using a bridge rectifier so that if the wires got swapped over, the device would still work. Depending on the application and the environment some people would use a DC-DC converter to provide high isolation levels between power in and actual PCB supply. This can help when your PCB is meant to be locally functionally earthed because, without isolation, your circuit might be subject to an earth fault current from some other piece of equipment - to obtain a single earth point means you should isolate your incoming supply.