Formula to compute output of a linear system
It's correct. Here's the general way how you can solve such a problem – in this specific example it would actually be easier to just solve the integral, but that seems indeed like homework you should be doing yourself.

I'll call the step function $\Theta$, that's how we always call it in physics – hope it doesn't confuse, and the convolution
$$
  (f\star g)(a) := \int_{\{(t,t&#39;)\colon t+t&#39;=a\}}\!\!\!\!\!\!\!\mathrm{d}(t,t&#39;)\;f(t)\cdot g(t&#39;) = \int_\mathbb{R}\!\mathrm{d}\tau\; f(\tau)\cdot g(t-\tau)
$$
which is just the definition that you use. We know the convolution theorem
$$
  \mathrm{FT}(f\star g)(\omega) = \frac1{\sqrt{2\pi}}\mathrm{FT}(f)(\omega)\cdot\mathrm{FT}(g)(\omega)
$$
(the factor $\frac1{\sqrt{2\pi}}$ depends on your definition of the Fourier transform). To apply this to your problem first calculate
$$
  \mathrm{FT}(h)(\omega) = \frac1{\sqrt{2\pi}}\int_\mathbb{R}\!\mathrm{d}t\; e^{-t}\Theta(t)\cdot e^{-i\omega t}
 = \frac1{\sqrt{2\pi}}\int\limits_0^\infty\!\mathrm{d}t\; e^{-t}\cdot e^{-i\omega t}
$$
since the Heaviside function cancels everthing for $t&lt;0$, then this is
$$
 = \frac1{\sqrt{2\pi}}\int\limits_0^\infty\!\mathrm{d}t\; e^{-(1+i\omega) t}
 = \frac1{\sqrt{2\pi}}\left.\left[\frac{e^{-(1+i\omega) t}}{-(1+i\omega)}\right]\!\!\right|_{t=0}^{\infty}
 = \frac1{\sqrt{2\pi}}\Bigl(\frac{0}1 - \frac{e^0}{-(1+i\omega)}\Bigr)
$$
which is just $=1/(\sqrt{2\pi}\cdot(1+i\omega))$. Meanwhile,
$$
  \mathrm{FT}(x)(\omega) = \mathrm{FT}(\lambda t.1)(\omega) + \tfrac12\mathrm{FT}(\lambda t.\cos(400\pi t))(\omega).
$$
These Fourier transforms only exist in a distributional sense; it is known (can easily be shown by inverse $\mathrm{FT}$) that
$$
  \mathrm{FT}(\lambda t.1) = \sqrt{2\pi}\;\delta
$$
and
$$
  \mathrm{FT}(\cos)(\omega) = \frac{\sqrt{2\pi}}{2}\bigl(\delta(\omega-1)+\delta(\omega+1)\bigr).
$$
So we have
$$\begin{aligned}
  \mathrm{FT}(y)(t) =&amp; \frac{\sqrt{2\pi}\bigl(\delta(\omega)+\tfrac14\delta(\omega/400\pi-1)+\tfrac14\delta(\omega/400\pi+1)\bigr)}{\sqrt{2\pi}(1+i\omega)}
  \\=&amp; \frac{\bigl(\delta(\omega)+\tfrac14\delta(\omega/400\pi-1)+\tfrac14\delta(\omega/400\pi+1)\bigr)}{1+i\omega}
\end{aligned}$$
and therefore
$$\begin{aligned}
  y(t) =&amp; \mathrm{IFT}(\ldots)(t)
   = \frac1{\sqrt{2\pi}}\int_\mathbb{R}\!\mathrm{d}\omega\; e^{i\omega t}\frac{\bigl(\delta(\omega)+\tfrac14\delta(\omega/400\pi-1)+\tfrac14\delta(\omega/400\pi+1)\bigr)}{1+i\omega}
 \\=&amp; \frac{e^0}{\sqrt{2\pi}\cdot 1} + \frac{e^{-400 i\pi t}}{\sqrt{2\pi}(1+\tfrac1{400\pi}i)} + \frac{e^{400 i\pi t}}{\sqrt{2\pi}(1-\tfrac1{400\pi}i)}
\end{aligned}$$
We could know calculate this as an exact expression of trigonometric functions by writing out the $1\pm\tfrac{i}{400\pi}$ as a polar complex number.