Kallmann syndrome (KS) is a rare genetic disorder manifested by the combination of hypogonadotropic-hypogonadism and olfactory dysfunction. It is usually diagnosed at 14 - 16 years of age due to delayed puberty. However, delays in diagnosis have been reported in a few cases. We presented a 34-year-old man presented with primary infertility as the chief complaint. Physical examination revealed bilateral gynecomastia, Tanner stage 2, and anosmia. Hormonal studies show a hypogonadotropic hypogonadism profile. Genetic testing revealed a normal male karyotype. Abdominal ultrasonography (USG) revealed a small prostate, and testicular USG demonstrated small testicles. Neuroimaging study revealed olfactory bulb agenesis and hypoplasia of the olfactory sulcus. Treatment was done by testosterone replacement therapy, and the patient is now on a regular follow-up. In conclusion, suspected clinical features of KS may guide the diagnosis with comprehensive hormonal and imaging studies. However, the reported patient was diagnosed extremely late.