How to address graduate school recommendation letter?
According to Wikipedia


Dear Sir or Madam or 'Sir/Madam' (If the gender of the reader is unknown).
To Whom It May Concern (If the writer wishes to exclude the gender of the reader from the salutation and/or to convey that the reader should forward the copy to one more suited to receive or respond appropriately).


As @Davidmh has mentioned it is least concerned. I personally have chosen to use "To Whom It May Concern" when I applied to grad school as in some grad schools the recommendation letters might get forwarded from the admissions staff to various professors. 

Edit : The first is British English and the second is American English as mentioned by @Davidmh