Invalid challenge-ciphertext in IND-CPA and IND-CCA games
$A$ outputs a bit depending on whether it thinks you gave it an encryption of $m_0$ or $m_1$.

If you cheat and give it something completely different it has no usable information but still needs to output a bit.

If your PKE scheme has invalid ciphertexts $A$ might be able to detect that or not depending on the specifics.

So if you want to exchange your encryption of $m_0$ or $m_1$ for something else in a proof you have to show that $A$ cannot distinguish between these two situations.

Intuitively you can think about it this way:
If $A$ had some way to distinguish between the real security game and your simulation it means that your simulation looks different. So how do you know that whatever $A$ did to attack the original game will work on your simulation if it doesn't look the same to $A$?