Why the BRDF of specular reflection is infinite in the reflection direction? I know the BRDF of specular reflection is nonzero only in the reflection direction.

But why it is infinite?

A paragraph on page 36 of Advanced Global Illumination:
In the BRDF curve, the total area under the curve is the albedo: what fraction of incident light is reflected in total (as opposed to being absorbed or transmitted). For a perfect reflector, the area under the curve sums to 1, because it reflects all of the incident light. This area limits how high the curve can be. For example, a perfect diffuse reflector has a BRDF that's a flat line at $\frac{1}{2\pi}$, because it reflects the light evenly across the whole hemisphere, the solid angle of a hemisphere is $2\pi$, and the area has to be 1.

In a perfect specular reflector, the area under the curve is still 1: it still reflects all of the incident light. But this time, it's reflecting all of that light in one direction. The BRDF curve is a single spike. If you draw a graph with an infinitely thin spike, and the area under the graph has to be 1, then the spike must also be infinitely high. It's the only way to make the area correct.

Obviously a real material can't be a perfect specular reflector, so it will have two differences. First, the area under the curve will be less than 1,  because some light is absorbed. This doesn't matter much to us. More importantly, the reflection peak can't be infinitely thin: the reflection will be blurred ever so slightly. This means the peak won't be infinitely high: the wider the peak, the less high it has to be to still have an area of 1.



The remark about "$\delta$-functions" is because of a function (properly speaking, it's not a function but a distribution) called the Dirac $\delta$ (delta). This distribution is an infinitely thin, infinitely tall spike at $x = 0$, whose area is 1, just like our perfect specular BRDF.