Bandwidth of a finite impulse response filter
Yes, the length of a filter limits the "frequency-selectiveness". If you remember modern physics, that's mathematically very related to the Heisenberg uncertainty principle: something that is temporally short can't be very sharp in the frequency domain, and vice versa.

You can find one example of such a constraint in this answer, though the formula only applies to specific filter constraint.

Generally, filter design is one of the oldest application fields of evolutionary / ML algorithms. You'll be delighted to find that even some of the very "classical", non-ML algorithms for bandpass filter design make extensive use of a loss / reward function while optimizing a filter.

If you're doing modern machine learning for filer design, there's a lot of modern literature, but I'd really start by reading up on the Remez exchange (or just Remez) algorithm, to give you a starting direction. Understanding that relatively simple algorithm and its convergence will give you a solid base!  

Especially when later writing a report on it, that would be the most basic benchmark for your ML approach that I could think of. Compare yourself to that!