what is window size?
That paper is very poorly written. It's no wonder you have a hard time understanding it; I certainly did!

What I think is happening is that several of the DCT coefficients are being thrown out (windowed). So if the window is 100%, then all the DCT coefficients are kept; if the window is 30% then 70% of the coefficients are thrown out (zeroed).

See Figure 11 of that paper:



The left hand signal are the DCT coefficients. The right-hand graph is the applied window. The two are multiplied together.