Vestigial Filter problem? I have been stuck on this question for a while now. It has to do with vestigial sideband. 



I wasn't sure if I should be dividing $H(\omega)$ graph values by 2 because only the positive side of the filter is given....

The answer that I am getting is:

We know that $VSB(t) = A_c f(t)\cos(\omega_c t) \ast h(t)$

then in frequency form I get: 

$$VSB(\omega) = A_c\pi  0.5 (\delta(\omega + (\omega_c + \omega_m)) + \delta(\omega + (\omega_c - \omega_m)) + \delta(\omega - (\omega_c + \omega_m)) + \delta(\omega - (\omega_c - \omega_m))) H(\omega)$$

Now using the graph I am multiplying the appropriate $H(\omega)$ values:

$$VSB(\omega) = A_c \pi 0.5 (0.25 \delta(\omega + (\omega_c + \omega_m)) + 0.75 \delta(\omega + (\omega_c - \omega_m)) + 0.25 \delta(\omega - (\omega_c + \omega_m)) + 0.75 \delta(\omega - (\omega_c - \omega_m)))$$

where carrier wave is given by $A_c \cos(\omega_c t)$

I don't have the solution to this, so I have no idea if my answer or reasoning is correct...
It would be sooooo helpful if someone could confirm/clarify
Your reasoning seems reasonable and correct to me. Assuming that the filter's impulse response $h(t)$ is real, then its magnitude response $\left|H(\omega)\right|$ will be symmetric about $\omega=0$, so the negative-frequency terms will be scaled by the same values as those on the positive side of the frequency axis. I don't see any reason why you would need to multiply or divide by 2 anywhere.

The only thing that might differ if you see a solution somewhere else might be the scaling factor at the front of the expression. The typical transform using angular frequency would be:

$$
\mathcal{F}\{\cos(ax)\} = \pi\left(\delta(\omega - a) + \delta(\omega + a)\right)
$$

which assumes a Fourier transform definition of:

$$
\mathcal{F}\{x(t)\} = X(\omega) = \int_{-\infty}^{\infty}x(t) e^{-j\omega t}dt
$$

It looks like you have an extra factor of $0.5$ at the front of your expression that you might have gotten from the transform. Note that different implementations or expressions of Fourier transforms often have arbitrary scaling factors included; what's most important is that if you apply the forward and inverse transforms in sequence, you get the original signal back:

$$
\mathcal{F}^{-1}\left\{\mathcal{F}\{x(t)\}\right\} = \mathcal{F}^{-1}\left\{X(\omega)\right\} = x(t)
$$

or at least to some known scaling factor.