A 77-year-old man was initially diagnosed with idiopathic pulmonary fibrosis (IPF) and treated with anti-fibrotic nintedanib. Despite undergoing anti-fibrotic treatment for one year, his condition remained unstable. The patient was admitted to our hospital for exertional dyspnea. We performed an exposure assessment, including 2-week antigen avoidance and an environmental inhalation challenge, and successfully re-diagnosed him with fibrotic hypersensitivity pneumonitis (HP), known as chronic farmer's lung. Adding oral glucocorticoids to the nintedanib treatment improved his condition. Although antigen avoidance and environmental inhalation challenge tests are not standardized, they may be useful for diagnosing fibrotic HP when properly applied.