How Is the MATLAB `filter()` Function So Fast?
Actually it seems MATLAB implementation of the filter() function is pretty straight forward and not fast.

For a fast implementation, have a look at FilterM by Jan Simon.

Update

In the latest releases of MATLAB (From R2016b and above) the performance of the filter() function has improved.
The metdhos to accelerate those operations are usually based on:  


Utilizing SIMD Operations (SSE / AVX in x86).
Multi Threading.
Loop Unrolling.