Q: Evaluate $\int_{-\infty}^{\infty}\frac{\sinh\left(y\sqrt{\alpha^2-\omega^2}\right)}{\sinh\left(H\sqrt{\alpha^2-\omega^2}\right)}e^{i\alpha x}d\alpha$ I need to evaluate the Fourier inverse integral
$\displaystyle \int_{-\infty}^{\infty}\frac{\sinh\left(y\sqrt{\alpha^2-\omega^2}\right)}{\sinh\left(H\sqrt{\alpha^2-\omega^2}\right)}e^{i\alpha x}d\alpha \tag*{}$
which arose while solving a PDE.
Here, $H>0,x\in\mathbb{R},y\in[0,H]$. The domain of $\omega$ was not given in the original problem, but I am going to assume $\omega>0$ for simplicity.
The problem asks us to introduce proper branch cuts for the square root function before evaluating the integral.
For reference, this is the original question. My attempt up until I get the integral is also shown in the link.
My Attempt
The branch points of the square root functions are at $\alpha =\pm\omega$. So, I considered the following branch cuts and contours.

(Here $\omega$ has an absolute value, but you can ignore it and assume $|\omega|=\omega$)
We, firstly, need to find the poles of the integrand in the upper half-plane. Those are given by the equation
$H\sqrt{\alpha^2-\omega^2}=n\pi i\tag*{}$
Solving this, we obtain
$\displaystyle \alpha =\pm\sqrt{\omega^2-\frac{n^2\pi^2}{H^2}}\tag*{} $
where $n=0,1,2,\cdots$.
The problem is that some of those poles are on the branch cuts depending on the parameters. I have been told this is not permissible, so I am not sure how to proceed.
Edit: The statement that "some of those poles are on the branch cuts" is not correct. It is on the real axis but between $[-\omega,\omega]$.
Edit2: Tables of Fourier Transforms and Fourier Transforms of Distributions by Fritz Oberhettinger states(p37,7.48) that if $a<b$, we have
$\displaystyle \int_{0}^{\infty} \frac{\sinh{(a\sqrt{k^2+x^2}})}{\sinh{(b\sqrt{k^2+x^2}})}\cos{xy}dx = -\pi b^{-1} \sum_{n=0}^{\infty}(-1)^nc_n\sin{(ac_n)}v_n^{-1}e^{-yv_n}$
where $c_n=n\pi/b$, $v_n = (k^2+c_n^2)^{1/2}$. I would guess our integral would have a similar form.

A: Even though the branch cut $(-\infty,0)$ of the square-root transfers to $\sinh(\sqrt{z})$, the integral actually does not possess any branch cut. The only branch-cut that could possibly occur, would be $(-\omega,\omega)$. However, the phase-jump cancels out in ratio: if for some $\Re(\alpha) \in (-\omega,\omega)$, $\alpha$ goes from the upper half-plane into the lower half-plane, $\sqrt{\alpha^2-\omega^2}$ changes from being of the form $\pm i\cdot r$ to $\mp i\cdot r$, for some real number $r$, but $$\sinh(\pm i r)=\pm i \sin(r) \, .$$
Hence, no jump i.e. no branch-cut.
Since you already know how to obtain the result from the residue theorem, here is a proof that shows the vanishing of the semicircle. However, you do not need to use a semi-circle. You can also take a box of side-length $R$, which turns out to be a bit easier.
Writing $\alpha=r+is$, the contour for $x>0$ is
$$C_R=\{r=R,s\in(0,R)\} \cup\{r\in(R,-R),s=R\} \cup \{r=-R,s=(R,0)\} \, ,$$
from which we'll find
$$\left|\int_{C_R}\frac{\sinh\left(y\sqrt{\alpha^2-\omega^2}\right)}{\sinh\left(H\sqrt{\alpha^2-\omega^2}\right)} \, e^{i\alpha x} \, d\alpha \right| \leq \int_{0}^R \left|\frac{\sinh\left(y\sqrt{(R+is)^2-\omega^2}\right)}{\sinh\left(H\sqrt{(R+is)^2-\omega^2}\right)} \right| \, e^{-xs} \, ds \\
+\int_{-R}^{R} \left|\frac{\sinh\left(y\sqrt{(r+iR)^2-\omega^2}\right)}{\sinh\left(H\sqrt{(r+iR)^2-\omega^2}\right)} \right| \, e^{-xR} \, dr + \int_{0}^R \left|\frac{\sinh\left(y\sqrt{(-R+is)^2-\omega^2}\right)}{\sinh\left(H\sqrt{(-R+is)^2-\omega^2}\right)} \right| \, e^{-xs} \, ds \\
= I_1 + I_2 + I_3 \, .$$
Now we notice:

*

*The first and third integral are the same, since the arguments of the absolute value function are just complex-conjugate to each other.

*As $|R+is|$ is large, i.e.
$$\sqrt{(R+is)^2-\omega^2} = R + is + O\left( \frac{1}{|R+is|} \right)$$
we can remove the square-root. The error made will be small compared to the main term, since $$\sinh(R+is+\epsilon) = \sinh(R+is) \left\{ \cosh(\epsilon) + \coth(R+is) \sinh(\epsilon) \right\} \\
= \sinh(R+is) \left\{1+O(\epsilon)\right\}$$
where $\epsilon=O\left(|R+is|^{-1}\right)$ is small.

*The following identity holds for all $x,y \in \mathbb{R}$
$$\left| \sinh\left(x+iy\right) \right|^2 = \sinh^2(x) + \sin^2(y) \, .$$

*In estimating the contribution along the box, we need to avoid the poles of $1/\sinh$ on the imaginary axis at $in\pi$. As $R\rightarrow \infty$, we can assume $HR=(n+1/2)\pi$ as $n \rightarrow \infty$, for integers $n$.

Using these observations, we can estimate as follows:
$$I_1 \leq \int_0^R \sqrt{\frac{\sinh^2(yR)+\sin^2(ys)}{\sinh^2(HR)+\sin^2(Hs)}} \, e^{-xs} \, {\rm d}s \leq \int_0^R {\frac{\cosh(yR)}{\sinh(HR)}} \, e^{-xs} \, {\rm d}s \leq \frac{\cosh(yR)}{\sinh(HR)} \frac{1}{x} \\
I_2 \leq \int_{-R}^R \sqrt{\frac{\sinh^2(yr)+\sin^2(yR)}{\sinh^2(Hr)+\sin^2(HR)}} \, e^{-xR} \, {\rm d}r \leq \int_{-R}^R {\frac{\cosh(yr)}{\cosh(Hr)}} \, e^{-xR} \, {\rm d}r \leq 2R e^{-xR} \, .$$

A: For completeness, I will provide the complete solution I can imagine.
We calculate the integral for $x>0$. The case $x<0$ can be solved with a similar method.
We firstly note that the poles of the integrand are located at $\alpha =\pm p_n(n=1,2,3\cdots)$ where $p_n= \sqrt{\omega^2-\frac{n^2\pi^2}{H^2}}$. If $\omega H<\pi$ then every pole is on the imaginary axis, but if $\omega H>\pi$ then some poles are on the real axis. We consider the case $\omega H<\pi$. Otherwise we can consider the contour going around the poles.
The integrand has a branch point at $\alpha=\pm \omega$, so we define a branch cut on a real axis going away from the branch points. We then consider the following rectangle contour as @Diger advises.

Here, $\epsilon>0$ is an arbitrarily small number, and $R$ is an arbitrarily large number (However, we take $R$ so that rectangle does not have poles on it). For $x>0$ we take the rectangle in the upper half-plane. For $x<0$ we use the dotted rectangle.
We now calculate the residues of the poles at $\alpha=p_n$.  We note that the residue of $\frac{N(z)}{D(z)}$ at $z=z_0$ is $\frac{N(z_0)}{D'(z_0)}$($N, D$ is a function that is analytic near $z=z_0$ and $D$ has a simple zero at $z=z_0$). Applying the formula, we obtain that the residue at $\alpha=p_n$ is
$\displaystyle -H^{-1}(-1)^n c_n\sin{(yc_n)}p_n^{-1}e^{ip_nx}\tag*{}$
Here, $c_n=n\pi /H$. Note that we have a similar formula in the question.
Thus, by the residue theorem, the integral on the rectangle is
$\displaystyle -2\pi iH^{-1}\sum_{|p_n|<R}(-1)^n c_n\sin{(yc_n)}p_n^{-1}e^{ip_nx}\tag*{}$
Now we calculate integrals on each contour. The integrals on $C_1, C_2$ and $C_3$ goes to $0$ as $R\to\infty$, as @Diger proves. So we need to prove that the integral on $E_1$ and $E_2$ goes to $0$ as $\epsilon \to 0$.
On $E_1$, we have $\alpha =-\omega +\epsilon e^{i\theta}(\theta:\pi\to 0)$. Thus, $d\alpha =i\epsilon e^{i\theta}$ and $K=(-2\omega \epsilon e^{i\theta}+\epsilon^2 e^{2i\theta})^{1/2} = -2\omega \epsilon e^{i\theta}+O(\epsilon^2)$. Thus, we have $K=-2\omega \epsilon \cos{\theta}-2i\omega \epsilon\sin{\theta}$.
We have $|\sinh(x+yi)|^2=\sinh^2(x)+\sin^2(y)$ and $\frac{4}{\pi^2}c^2\leq \sin^2{c}\leq c^2\leq \sinh^2{c}$ for small $c$, so

(Forgive the image; I got too lazy)
The integral on $E_2$ also goes to zero by the same method. Thus, by taking $\epsilon\to 0, R\to\infty$, we have
$\displaystyle \int_{-\infty}^{\infty}\frac{\sinh\left(y\sqrt{\alpha^2-\omega^2}\right)}{\sinh\left(H\sqrt{\alpha^2-\omega^2}\right)}e^{i\alpha x}d\alpha = \displaystyle -2\pi iH^{-1}\sum_{n=1}^{\infty}(-1)^n c_n\sin{(yc_n)}p_n^{-1}e^{ip_nx}\tag*{}$
If $\omega H<\pi$, every $p_n$ is imaginary number, so we can write $p_n=iv_n$ where $v_n=\sqrt{n^2\pi^2/H^2-\omega^2}$. Thus we have
$\displaystyle \int_{-\infty}^{\infty}\frac{\sinh\left(y\sqrt{\alpha^2-\omega^2}\right)}{\sinh\left(H\sqrt{\alpha^2-\omega^2}\right)}e^{i\alpha x}d\alpha = \displaystyle -2\pi H^{-1}\sum_{n=1}^{\infty}(-1)^n c_n\sin{(yc_n)}v_n^{-1}e^{-xv_n}\tag*{}$
which gives the exact same formula like the one in the question.
