Increased collagen expression in tumors is associated with increased risk of metastasis, and triple-negative breast cancer (TNBC) has the highest propensity to develop distant metastases when there is evidence of central fibrosis. Transforming growth factor-beta (TGF-beta) ligands regulated by cancer-associated fibroblasts (CAFs) promote accumulation of fibrosis and cancer progression. In the present study, we have evaluated TNBC tumors with enhanced collagen to determine whether we can reduce metastasis by targeting the CAFs with Pirfenidone (PFD), an anti-fibrotic agent as well as a TGF-beta antagonist. In patient-derived xenograft models, TNBC tumors exhibited accumulated collagen and activated TGF-beta signaling, and developed lung metastasis. Next, primary CAFs were established from 4T1 TNBC homograft tumors, TNBC xenograft tumors and tumor specimens of breast cancer patients. CAFs promoted primary tumor growth with more fibrosis and TGF-beta activation and lung metastasis in 4T1 mouse model. We then examined the effects of PFD in vitro and in vivo. We found that PFD had inhibitory effects on cell viability and collagen production of CAFs in 2D culture. Furthermore, CAFs enhanced tumor growth and PFD inhibited the tumor growth induced by CAFs by causing apoptosis in the 3D co-culture assay of 4T1 tumor cells and CAFs. In vivo, PFD alone inhibited tumor fibrosis and TGF-beta signaling but did not inhibit tumor growth and lung metastasis. However, PFD inhibited tumor growth and lung metastasis synergistically in combination with doxorubicin. Thus, PFD has great potential for a novel clinically applicable TNBC therapy that targets tumor-stromal interaction.