 Speculation has also surrounded Benjamin Cardozo , whose celibacy suggests repressed homosexuality or asexuality . The fact that he was unmarried and was personally tutored by the writer Horatio Alger ( alleged to have had sexual relations with boys ) led some of Cardozo 's biographers to insinuate that Cardozo was homosexual , but no real evidence exists to corroborate this possibility . Constitutional law scholar Jeffrey Rosen noted in a New York Times Book Review of Richard Polenberg 's book on Cardozo : 
