Working memory is an executive memory process that allows transitional information to be held and manipulated temporarily in memory stores before being forgotten or encoded into long-term memory. Working memory is necessary for everyday decision making and problem solving, making it a fundamental process in the daily lives of older adults. Working memory relies heavily on frontal lobe structures and is known to decline with age. The current study aimed to determine the neural correlates of decreased working memory performance in the frontal lobes by comparing cortical thickness and cortical surface area from two demographically matched groups of healthy older adults, free from cognitive impairment, with high versus low N-Back working memory performance (N = 56; average age = 70.29 +/- 10.64). High-resolution structural T1-weighted images (1 mm isotropic voxels) were obtained on a 3T Philips MRI scanner. When compared to high performers, low performers exhibited significantly decreased cortical surface area in three frontal lobe regions lateralized to the right hemisphere: medial orbital frontal gyrus, inferior frontal gyrus, and superior frontal gyrus (FDR p < 0.05). There were no significant differences in cortical thickness between groups, a proxy for neurodegenerative tissue loss. Our results suggest that decreases in cortical surface area (a proxy for brain structural integrity) in right frontal regions may underlie age-related decline of working memory function.