Design of a Complex FIR Filter For apply least-squares linear-phase FIR filter design,with frequency domain specification is not symmetrical.

The pass-band error function,

$$E(\mathbf{h})_p=\int_{\omega_{p_1}}^{\omega_{p_2}}| \mathbf{c}^T(\omega)\cdot \mathbf{h}-D(\omega)|^2d\omega \tag{1}$$

The stop-band error function,

$$E(\mathbf{h})_s=\int_{\omega_{s_1}}^{\omega_{s_2}}|\mathbf{c}^T(\omega)\cdot \mathbf{h}|^2d\omega\tag{2}$$

h: unknown complex coefficients.

Total error function,

$$E(\mathbf{h})_t=w_1E(\mathbf{h})_p+w_2E(\mathbf{h})_s\tag{3}$$

$w_1,w_2$: weighting constants .


The equation $(1)$ : which represents a filter approximation by determine the error between actual and desired response in pass-band , why the method is different in stop-band (equation $(2)$)  ?


-In matlab ,does it the freqz function achieve the transpose operation ?
In the stopband(s), Eqs $(1)$ and $(2)$ are equivalent, because in the stopband the desired response equals zero: $D(\omega)=0$. The reason why you might want to split the error in passband and stopband error is to apply different weights, as shown in your Eq. $(3)$.

The function freqz computes the frequency response of a discrete-time filter on a grid of frequencies, given the filter coefficients. I'm not sure what you mean by "achieve the transpose operation".