Invasive pulmonary aspergillosis (IPA) usually occurs in patients with severe immunodeficiencies involving neutropenia. Underlying lung disease is a well-known risk factor of IPA; however, interstitial lung disease has not been recognized as a risk factor of IPA. We herein report a patient with fibrotic nonspecific interstitial pneumonia who experienced IPA without neutropenia. His IPA was fatal and showed unusually slow disease progression over one month. The computed tomography findings showed only nonspecific consolidation and no typical lesions suggestive of IPA. Finally, the autoptic findings revealed numerous Aspergillus fungi, neutrophilic pulmonary necrosis, and vessels invaded by Aspergillus fungi.