How to setup multisite to multisite rsync with drush?
On Drupal 6, your sites folders must be the same.  Since you are using Drupal 7, they can be different, but it is more convenient if they are the same.
You do not need to copy all of your multisites to your dev server.  The drush rsync option you are looking for is --exclude-other-sites.  This will cause rsync to skip everything in the sites folder except for sites/all, and the site you are syncing from.
sites/all is where sites/all/modules and sites/all/themes go, so you typically need these.  With some effort it might be possible to move your contrib modules elsewhere (if everything was in a profile, or if you hand-modified your registry), but this would be an unusual setup.  sites/default is not needed.