But this point of view is misleading, because the diagrams not only describe scattering, but they also are a representation of the short-distance field theory correlations.
The number of Feynman diagrams at high orders of perturbation theory is very large, because there are as many diagrams as there are graphs with a given number of nodes.