drush "The drush command 'self-update' could not be found."
self-update was removed a while ago. I'm not sure what the official replacement is, but a simple git pull; composer install does the job for me with the latest version (run it in the drush lib folder, probably in /usr/local/Cellar if it was brewed).

theme list (and theme) don't appear in the list of available commands (although self-update does, ironically). Maybe you're looking for 

drush pm-list --type=theme