Multi-site - Setup remote asset volume to store assets on a per site/folder basis
You do this by adding {site.handle} to the fields upload location. 

This will create the subfolders like you mentioned. It will also create a default subfolder for the default site, but that's ok.

Note: Make sure to rebuild your Asset Indexes in Utilities > Asset Indexes after you add the subfolder path. 

That should ensure that they point to the subfolder next time you go to upload an image. I had trouble with it still pointing to the root volume directory, but that fixed it.

Also, all your current images will still work, but they will no longer show when adding/editing assets (As they are still in the root folder) You will need to copy these to the appropriate subfolder and rebuild your asset indexes (or upload again) if you want them to appear. Don't delete them from the root folder until you have updated the entries that access them.

I haven't been able to find a way to dynamically change the volumes. As far as I know it is not possible, but using subfolders works well.