Any process that stops, prevents or reduces the frequency, rate or extent of response to macrophage colony-stimulating factor.
down-regulation of response to macrophage colony-stimulating factor stimulus