Inverse Agonist vs Neutral Antagonist
Many receptors have a natural constitutive, or "baseline", amount of response that they generate, they're not just an on-off switch. At baseline, many receptors will generate a partial response, which means that certain things binding to them can either increase or decrease their response. 

A neutral antagonist is one that causes no change in the signaling response of the thing that it's binding to. It simply stays the same. Note that this can still be important, because a neutral antagonist can block the binding of other agonists, ones that would have generated a change in response, even if the antagonist itself doesn't cause a response.

An inverse agonist is something that decreases the response of the receptor from its baseline all by itself. Often, this is caused by stabilizing the receptor in it's "off" state. If the receptor has some degree of constitutive activity, an inverse agonist will decrease its activity.

See this graph from this wikipedia page for a good explanation: