Castrated men â '' eunuchs â '' were often admitted to special social classes and were used particularly to staff bureaucracies and palace households : in particular , the harem . Castrated men -- eunuchs -- were often admitted to special social classes and were used particularly to staff bureaucracies and palace households : in particular , the harem .