HLA-A is a locus of the major histocompatibility complex situated on chromosome 6p21.3. HLA-A has been shown to be associated with susceptibility to Alzheimer's disease (AD). In this study, we firstly investigated the association of gene variants in HLA-A and brain structures on MRI in a large sample from the Alzheimer's Disease Neuroimaging Initiative (ADNI) to explore the effects of HLA-A on AD pathogenesis. We selected the hippocampus, parahippocampus, posterior cingulate, precuneus, middle temporal, entorhinal cortex, and amygdala as regions of interest (ROIs). In hybrid population analysis, our results showed a marginally significant association between rs9260168 and the atrophy of the left parahippocampus (P = 0.007, Pc = 0.054), rs3823342 and the atrophy of the left parahippocampus (P = 0.014, Pc = 0.054), rs76475517, which only exists in Caucasians with HLA-A23 or HLA-A24 alleles, and the atrophy of the right amygdala (P = 0.010, Pc = 0.085) at baseline. In particular, the haplotype (TGACAAGG), as a surrogate marker of HLA-A2, was founded to be positively associated with the atrophy of the right hippocampus (P = 0.047) at baseline. Furthermore, we detected the above four associations in mild cognitive impairment (MCI) subpopulation analysis. Our study provided preliminary evidences supporting HLA-A2 in Caucasians contribute to the risk of AD by modulating the alteration of hippocampal volume and HLA-A gene variants appear to play a role in altering AD-related brain structures on MRI.