How do bode plots work with unstable systems work?
The frequency response can exist, even if there are poles in the right half-plane, as long as there are no poles on the $j\omega$-axis. You cannot use the unilateral Laplace transform, instead you must use the bilateral Laplace transform (with the lower integration limit at $-\infty$), or, simply, the Fourier transform.

The corresponding time domain function is left-sided (or two-sided, if there are also poles in the left half-plane), i.e., non-causal. Note that poles in the right half-plane only indicate instability if you restrict yourself to causal systems. Non-causal systems can be stable if there are poles in the right half-plane.

Take as an example two simple first order systems, one with a pole in the left half-plane, and one with a pole in the right half-plane, such that it is a mirror image of the left half-plane pole:

$$H_1(s)=\frac{1}{s+1}\tag{1}$$
$$H_2(s)=\frac{1}{s-1}\tag{2}$$

If $H_1(s)$ is the frequency response of a causal system (note that we can decide that in this case), then it is stable, and its frequency response is simply

$$H_1(j\omega)=\frac{1}{j\omega+1}\tag{3}$$

In the same way, if we decide that $H_2(s)$ describes a non-causal but stable system, then its frequency response exists and is given by

$$H_2(j\omega)=\frac{1}{j\omega-1}\tag{4}$$

Note that their Bode magnitude plots are identical, because

$$\big|H_1(j\omega)\big|=\big|H_2(j\omega)\big|=\frac{1}{\sqrt{1+\omega^2}}\tag{5}$$

Their phase plots are of course different.