Why is it desirable in an amplifier to have high input impedance and low output impedance?
Actually, the premise of your question is only true if the signals you are interested in are voltages. In that case, if the amplifier draws no current through its input (has infinite, or at least very high input impedance), then connecting it to a source won't affect the signal voltage, regardless of what the source impedance is.

Similarly, when you connect a load to the output of your amplifier, if the amplifier has zero output impedance, the signal voltage won't change, regardless of the current drawn by the load.

These properties make it much easier to analyze the behavior of the system overall.

However, if the signals you're interested in are currents rather than voltages, you want your amplifier to have zero input impedance and infinite output impedance for the same reasons.