FFT result of windowed cosine function
There is no problem in your plot if by fft you mean DTFT (or Discrete Time Fourier transform) and it is true to say &quot;the points on this dash line are the DTFT result of corresponding frequency&quot;:
If your function is a cosine:
$$
x[n] = \cos(\omega_0n) \Longrightarrow X(e^{j\omega}) = \pi\delta(\omega-\omega_0)+\pi\delta(\omega+\omega_0)\ ; \qquad|\omega|&lt;\pi
$$
and your window function is defined as follow:
$$
w[n] = \left\{
        \begin{array}{ll}
            1 &amp; \quad |n| \leq N \\
            0 &amp; \quad |n| &gt; N
        \end{array}
    \right.
\Longrightarrow W(e^{j\omega}) = \frac{\sin\left(\omega(\frac{N}{2}+1)\right)}{\sin\left(\frac{\omega}{2}\right)}
$$
as you mentioned if we multiply two functions in time domain, it's equal to periodic convolution in frequency domain:
$$
y[n] = x[n]w[n]\\ \Longrightarrow Y(e^{j\omega})=X(e^{j\omega})\circledast W(e^{j\omega})=\frac{1}{2\pi}\int_{2\pi}^{}X(e^{j\omega})W(e^{j(\omega-\theta)})  d\theta
$$
in this case (in one period):
$$
Y(e^{j\omega})=\frac{1}{2}\frac{\sin\left((\omega-\omega_0)\left(\frac{N}{2}+1\right)\right)}{\sin\left(\frac{\omega-\omega_0}{2}\right)}+\frac{1}{2}\frac{\sin\left((\omega+\omega_0)\left(\frac{N}{2}+1\right)\right)}{\sin\left(\frac{\omega+\omega_0}{2}\right)}
$$
if you plot this function in one period for $\omega_0=\frac{\pi}{5}$ and $N=2$ it look like this:

but if you change the window length to $N=4$ the result changes to:

and the cause for this, is that two main lobes overlap with one another.