Is there a downside to trying multiple flight instructors?
No. Definitely try more than one CFI and ideally, more than one school. This is the smart choice and one I recommended regularly when I was an active flight school CFI. 

You're about to spend a ton of money - do the due diligence and spend more up front to make sure you don't waste time and money with someone whose teaching style or personality clashes with you as a student.