Clearing Boost cache with drush
Answering my own question. I was missing something. 

Boost is a bit more fussy about the URL than the rest of drush when clearing the cache

drush --uri=http://my.site.name cc all 


will work.