Help with finishing this integral, to obtain the power spectral density of a pure cosine wave
You don't need to make it so complicated as to use two variables $t$ and $t^\prime$.

\begin{align} X_T(f) &amp;= \int_{-T/2}^{T/2} x(t)\exp(-j2\pi ft)\,\mathrm dt\\
&amp;= \int_{-T/2}^{T/2}\frac{\exp(j2\pi f_0t)+\exp(-j2\pi f_0t)}{2}\exp(-j2\pi ft)\,\mathrm dt\\
&amp;= \frac 12\int_{-T/2}^{T/2} \exp(j2\pi (f_0-f)t)+\exp(-j2\pi (f_0+f)t)\,\mathrm dt\\
&amp;= \frac 12 \left[\frac{\exp(j2\pi (f_0-f)t)}{j2\pi(f_0-f)}~ + ~\frac{\exp(j2\pi (f_0+f)t)}{j2\pi(f_0+f)}\right\vert_{-T/2}^{T/2}\\
&amp;= \frac 12 \left[\frac{\exp(j\pi (f_0-f)T)-\exp(-j\pi (f_0-f)T)}{j2\pi(f_0-f)}\right.\\
&amp;\qquad\qquad + \left.\frac{\exp(j\pi (f_0+f)T)-\exp(-j\pi (f_0+f)T)}{j2\pi(f_0-f)}\right]\\
&amp;= \frac T2 \big(\operatorname{sinc}((f_0+f)T) ~+ ~\operatorname{sinc}((f_0-f)T)\big)\\
&amp;= \frac T2 \big(\operatorname{sinc}((f-f_0)T) ~+ ~\operatorname{sinc}((f+f_0)T)\big).
\end{align}
So we get \begin{align}\frac 1T\big|X_T(f)\big|^2 &amp;= \frac T4 \big(\operatorname{sinc}^2((f-f_0)T) ~+ ~(\operatorname{sinc}^2((f+f_0)T)\\
&amp;\qquad\qquad + 2\operatorname{sinc}((f-f_0)T)(\operatorname{sinc}((f+f_0)T)\big)
\end{align}
In the frequency domain, the two $\operatorname{sinc}^2$ functions are of height $\frac T4$, are centered at $\pm f_0$ with central lobes of width $\frac 2T$ Hz. As $T \to \infty$, the central lobe widths shrink to $0$ and it is straightforward to show that for any value of $f$ other than $\pm f_0$,
$$\lim_{T\to\infty} \frac 1T\big|X_T(f)\big|^2 =0, ~ f \neq \pm f_0.$$
Now, $\frac 1T|X_T(\pm f_0)|^2$ diverges to $\infty$ and with the usual trick of pulling a Dirac delta out of the hat in such cases, we get that it must be that 
$$S_{xx}(f) = \lim_{T\to\infty} \frac 1T|X_T(f)|^2 = \frac 14 \big(\delta(f-f_0) + \delta(f+f_0)\big), ~~ \text{when} ~x(t) = \cos(2\pi f_0 t).$$