Confusion in cut-off frequency of LTI System
The shown fuction with a remarkable peaking within the passband looks like a Chebyshev-lowpass filter. For such a filter the cut-off frequency is commonly defined using the &quot;ripple&quot; and NOT the 3dB criterion.
That means: The end of the passband is defined at the point where the magnitude crosses the value for DC (in your case: &quot;1&quot;) for the last time.
The 3dB criterion is applied for first-order filters and for all other responses without peaking near the pole frequency (without any ripple).