Many studies suggest sex differences in memory and hippocampal size, and that hormone therapy (HT) may positively affect these measures in women; however, the parameters of HT use that most likely confer benefits are debated. We evaluated the impact of sex and postmenopausal HT use on verbal learning and memory and hippocampal size in 94 cognitively intact women and 49 men. Using analysis of covariance that controlled for age and education, women had better total word learning and delayed verbal memory performance than men. HT analyses showed that non-HT users performed similarly to men, while HT users performed better than men in Delayed Memory regardless of whether use was current or in the past. Women had larger hippocampal volumes than men regardless of whether they were HT users. Using univariate linear models, we assessed group differences in the predictive value of hippocampal volumes for verbal learning and memory. Hippocampal size significantly predicted memory performance for men and non-HT users, but not for HT users. This lack of relationship between hippocampal size and verbal learning and memory performance in HT users suggests HT use may impact memory through extra-hippocampal neural systems.