Downsides of switching Master to Simple recovery? I've read a lot about this over the last couple of days, but none specifically on the Master db.

If I switch from full to simple, will it flush out any plans, statistics, etc?
Any performance hits?

I know how to proceed, just never seen Master on full recovery before.

TIA
It does not really make sense to have the master database in FULL recovory. The installation process leaves the database in SIMPLE recovery, so this means that somebody changed it afterwards because they could. 

I do not think that you perform operations on the master database that require point-in-time recovery and if you do I strongly suggest that you rethink this strategy. Also, take into account that a database in FULL recovery needs to back up its log regularly in order to avoid filling the log files. Is this happening? How are you planning to restore these log backups in case?

That said, I would change it back to SIMPLE without worrying too much about performance hits.