Why is discrete cosine preferred to FFT in neuroimaging GLM
Say we have a signal,



The DFT assumes the signal is periodic, e.g.,



While the typical DCT assumes the signal is even at the boundaries, e.g.,



In the DFT example you can see there is a big jump from one side of the signal to the other. Since it assumes a periodic signal, this big jump introduces a lot of high frequency components. Thus the energy of the signal is spread out over more frequencies. For example, the log of the absolute values of the DFT is



Because the DCT assumes the signal is even at the boundaries, generally there is no big jump and thus no extra high frequency components. If this assumption is correct, more of the energy is concentrated in the lower frequencies. For example, the log of the absolute values of the DCT is 



Actually, point at which the DCT assumes the signal is even can be moved. It doesn't have to be at the boundaries. However, if your signal generally has some large ramp component, then it makes sense to use the boundaries.

For the edge condition the DCT is equivalent (with multiplication by a scalar) to the DFT of the mirrored signal,



The log of the absolute values of the DFT of this signal is



which is the same as the DCT result above.

Note

The DCT doesn't automatically reduce boundary effects. Consider the extreme example of a sine signal, its DFT and DCT:



Because the even assumption didn't hold, the DCT actually resulted in many more high frequency components.

Why use it

I'm guessing because of drift in the nueroimaging signal, it is not periodic, and thus there would be a big jump from start to end. This would introduce lots of high frequency components with the DFT. Using the DCT assumes the signal is even and reduces the boundary effects.