Invertiblity of LTI System
Your reasoning can bring you to the answer, but it's not spot on. Consider:
$$
x(t) = \delta(t)
$$
$$
h(t) = \delta(t-\tau)
$$
$$
y(t) = x(t)*h(t) = \delta(t-\tau)
$$

System output to an impulse at \$t = 0\$ is another impulse at \$t = \tau\$ (a time delay of \$\tau\$). If the system is LTI, then the following holds true:

$$
x(t) = \delta(t+\tau)
$$
$$
h(t) = \delta(t-\tau)
$$
$$
y(t) = x(t)*h(t) = \delta(t)
$$

Because the impulse now happens at \$t = -\tau\$, system output now happens at \$t = 0\$.