Fronto-limbic brain networks involved in regulation of impulsivity and aggression are abnormal in Borderline Personality Disorder (BPD). However, it is unclear whether, or to what extent, these personality traits actually modulate brain responses during cognitive processing. Using fMRI, we examined the effects of trait impulsivity, aggression, and depressed mood on regional brain responses in 31 female BPD and 25 control subjects during a Go No-Go task using Ekman faces as targets. First-level contrasts modeled effects of negative emotional context. Second-level regression models used trait impulsivity, aggression and depressed mood as predictor variables of regional brain activations. In BPD, trait impulsivity was positively correlated with activation in the dorsal anterior cingulate cortex, orbital frontal cortex (OFC), basal ganglia (BG), and dorsolateral prefrontal cortex, with no areas of negative correlation. In contrast, aggression was negatively correlated with activation in OFC, hippocampus, and BG, with no areas of positive correlation. Depressed mood had a generally dampening effect on activations. Effects of trait impulsivity on healthy controls differed from effects in BPD, suggesting a disorder-specific response. Negative emotional context and trait impulsivity, but not aggression or depression, diminished task performance across both groups. Negative emotional context may interfere with cognitive functioning in BPD through interaction with the neurobiology of personality traits.