Background: To delay cognitive decline, it is important to engage actively in preventive activities from middle age (50-64 years of age). We have developed a cognitive intervention program using picture book reading, and demonstrated that it is effective for improving memory in older adults. However, the effect of the intervention on memory and other cognitive functions in middle-aged people has not been examined. The current study investigated the effects of the picture book reading program on cognitive function in middle-aged people. Methods: This single-blind (examiners were blinded) randomized controlled trial was conducted in Tokyo, Japan. A total of 65 participants were randomly assigned to the intervention group (IG) (n = 32), in which members attended 12 picture book reading classes held once a week, or the active control group (CG) (n = 33), in which members received lectures on health maintenance. Cognitive tests were conducted before and after the intervention. The primary outcome was memory, and the secondary outcomes were verbal function and executive function. Results: The results showed that there was no significant difference between the IG and the CG in change scores (post minus pre) for memory. On the other hand, there was a significant difference in change scores of the category fluency, which is a measure of verbal function, suggesting improvements in IG compared to CG. There were also no significant differences in executive function. Conclusions: The results indicated that our previous finding of an improvement in memory function in older adults was not found in middle-aged people. However, the findings suggest that the picture book reading program may affect lexical access ability in verbal function among middle-aged people. Because maintaining verbal function is important for daily communication, these findings suggest that this program may be a useful countermeasure for cognitive decline in middle-aged people. Clinical Trial Registration: University Hospital Medical Information Network Clinical Trial Registry, https://upload.umin.ac.jp/cgi-open-bin/ctr_e/ctr_view.cgi?recptno=R000048012, Identifier: UMIN 000042071.