Shortlisting criteria for an oversubscribed course: first-come-first-served vs grade point and classroom diversity
The main problems with doing something unusual are that it is unfamiliar to your students, which could cause consternation if they feel they didn't receive enough info about how to get into your class, especially if it caused problems with their graduation plans.
At universities I'm familiar with, registration is allowed for grad students first, followed by seniors, then juniors, down the line. It would be really unfortunate to choose a third-year for the last spot and force a senior to take another year or semester because they couldn't get into your class because they had incomplete information.
&quot;Instructor discretion&quot; sounds like a recipe for mishandling individual qualifications, whether you want to talk about inherent bias or lack of information (GPA seems far too coarse for this).
A couple of simple ways to address your problem:

Split grad students into a separate grad student class that meets at the same time with the undergrads.
If you can in your registration system, reserve spots for non-seniors (common at universities I'm familiar with). While within-year, it would still be first-come-first-served, you would achieve your goal of having both third- and fourth-years.

Finally, I think having an application system would be OK, but it needs to be handled transparently and early so students can plan their schedule; i.e. your application is due early enough so that you can select your students before the earliest registration date. Ideally, you should have a second person help you evaluate. This also lets you ask for cover letters and CVs. You should be clear you are reserving, say, 10 spots for third-years. Then, you just lock registration and give codes to the students whom you selected.