Drush rsync permission woes (files directory)
Permissions issues are fairly complicated.

First off, the "failed to set times" message occurs because only the file owner can perform this operation.  The solution to this problem, then, is to insure that the 'remote-user' element of your Drush site alias is the same as the owner of the files on the target machine.  When pulling files to the local machine with drush rsync, it works best if you run Drush as the same user that the files are owned as.

What specific sets of permissions you choose after that is largely up to you, as there are a number of options that will work fine.  What I do is make all web server code files owned by a 'www-admin' user on the live site, and owned by my local user on my dev site.  On both live and dev, the files group is the webserver group, usually www-data, with directories that the webserver can write to being group-writable (and all others are set to not be group-writable, of course).  Any file created by the webserver will by necessity be owned by the webserver, of course.

This works well for most situations; however, it does mean that if you rsync %files from dev to live, you won't be able to overwrite anything that was uploaded by the webserver.  You might consider this a useful feature, though, as it prevents accidental overwrites.