What are the properties of continuous-time improper systems?
Note that the imaginary axis is not inside the ROC because $|H(j\omega)|\to\infty$ for $\omega\to\infty$. So the system is definitely unstable, regardless of causality, which confirms your intuition.

Concerning causality, the ROC is neither a right half-plane nor a left half-plane (both because of the pole at $s\to\infty$), which is consistent with a (generalized) function concentrated at $t=0$.