what is the theory behind Butter function?
The trigonometric functions enter the calculation because of the use the bilinear transform for transforming an analog filter to a discrete-time filter. The bilinear transform warps the frequencies of the analog filter, that's why we have to pre-warp the frequencies of the analog filter in order to obtain the desired cut-off frequencies of the discrete-time filter.

The frequency warping of the bilinear transform is described by the following equation:

$$\Omega=\tan\left(\frac{\omega}{2}\right)\tag{1}$$

where $\Omega$ is the frequency (in radians) in the analog domain, and $\omega$ is the normalized frequency (in radians) in the discrete-time domain. Eq. $(1)$ is used to pre-warp the specified frequencies to obtain the edge frequencies for designing the analog prototype filter. After applying the bilinear transform to that analog prototype, the resulting discrete-time filter will have the correct edge frequencies.

The design procedure is as follows:


Pre-warp the given edge frequencies according to $(1)$.
Design an analog band pass filter with the pre-warped edge frequencies by applying a frequency transformation to a low pass prototype filter.
Apply the bilinear transform to the analog band pass filter to obtain the desired discrete-time filter.