Analytical design of a bandstop Butterworth filter
it looks like you are designing a parallel highpass and lowpass filter.  i don't think that's the best way to do it.  instead, take your butterworth LPF prototype

$$\Big|H(j\Omega)\Big|^2 = \frac{1}{1+\Omega^{2N}}$$

and make this substitution for normalized $\Omega$:

$$ j\Omega = s \leftarrow \frac{Q}{\frac{s}{\Omega_0} + \frac{\Omega_0}{s}} $$

$Q$ is inversely proportional to the bandwidth of the bandstop filter  (if $N=1$, then $Q$ is the actual "Q" of the second-order filter.  $\Omega_0$ is the angular frequency of where the bandstop is tuned to.