When should I use the private file system instead of public file system?
You should use private file system for files to take under control of your drupal installation. 

If you have some content with files and pictures you want to be visible only for certain users or groups, then you should save it in private file system. Perhaps paid content or sensitive content. 

Content and images which is public to all do not need to be in private file system. For that it is better to use public file system to save server resources. 

All files in public space are usable for everyone who knows the url. 
Files in private space are not. You can get them only if drupal serves it to you.