People can reliably distinguish the sex of faces across age groups. Rates of accuracy are lower for infants, however, likely because they lack the pronounced sexually dimorphic features that develop during puberty. Given that previous research has shown that perceivers categorize adult sex automatically, we wondered whether this would extend to the faces of infants for whom sex is less legible. We tested this using a semantic priming paradigm in which infant faces preceded the categorization of stereotypically male and female names. Results showed that participants categorized the sex of male names significantly faster following perceptions of male versus female infant faces (though female faces did not significantly facilitate the processing of female names). The asymmetry in interference for male but not female faces supports evidence for a male default in conceptions of sex among infants previously found for adults. Individuals may therefore process sex automatically in the absence of overt cues (e.g., post-pubertal sexually dimorphic features or stereotypical clothing), providing additional evidence for the depth and flexibility of social categorization.