Aggressive behavior is assumed to be associated with certain patterns of social information processing. While some theories link aggression to a tendency to interpret ambiguous stimuli as hostile (i.e., enhanced sensitivity to anger), others assume an insufficient ability to perceive emotional expressions, particularly fear. Despite compelling evidence to support both theories, no previous study has directly investigated the predictions made by these two accounts in aggressive populations. The aim of the current study was to test processing patterns for angry and fearful facial expressions in violent offenders (VOs) and healthy controls (CTLs) and their association with self-reported aggression and psychopathy scores. In Experiment 1, we assessed perceptual sensitivity to neutral-emotional (angry, fearful, happy) blends in a task which did not require categorization, but an indication whether the stimulus is neutral or emotional. In Experiment 2, we assessed categorization performance for ambiguous fearful-happy and angry-happy blends. No group differences were revealed in Experiment 1, while Experiment 2 indicated a deficit in the categorization of ambiguous fearful blends in the VO group. Importantly, this deficit was associated with both self-reported psychopathy and aggression in the VO, but not the CTL group. The current study provides evidence for a deficient categorization of fearful expressions and its association with self-reported aggression and psychopathy in VOs, but no support for heightened sensitivity to anger. Furthermore, the current findings indicate that the deficit is tied to categorization but not detection stages of social information processing.