Is inverse matched filter useful?
In your question, you postulate the existence of two filters, let's say $p(t)$ and $g(t)$, such that $p(t) \star g(t)=\delta(t)$. This assumption is problematic because it implies that $G(f)=1/P(f)$, which requires $P(f) \neq 0$ for all $f$. Furthermore, for small values of $P(f)$, $G(f)$ will take arbitrarily large values (see Dilip Sarwate's comments above and below this answer).

Assuming (for the sake of discussion) the existance of such a pair of filters, there may not be any advantage to using them. I'll describe a possible application from digital communications. Let's say you want to transmit a number $A$ over an analog channel. You choose an appropriate analog pulse $p(t)$ and transmit $$s(t) = A\delta(t)\star p(t).$$ (The pulse $p(t)$ could be chosen to fit the channel bandwidth, and/or to have a certain energy). The receiver could perform the "inverse filter" operation on the signal $s(t)$: $$s(t) \star g(t)=A\delta(t),$$ where $g(t)$ is your "inverse matched filter". However, in practice it turns out that detecting $A$ in this way is not optimal, because it does not have the best signal-to-noise ratio. In other words, when you add noise to the transmitted signal, the received signal becomes $$r(t)=s(t)+n(t).$$ In this case, filtering $r(t)$ with the "inverse matched filter" $g(t)$ results in a worse signal-to-noise ratio than filtering with the filter matched to $p(t)$.

If you have access to it, I highly recommend "An Introduction to Matched Filters", by G. Turin, IRE Transactions on Information Theory, June 1960. Your question is answered in page 318.