Use MySQL to match input against saved queries
Invert.  Hmmm...  Indexing is about inverting.  Let's see if we can apply that concept here.

I see two types of tests in your example -- "set" operations (color) and "range" operations (size).

Suppose we could build the following into some kind of table?

blue  --&gt; Bob, Alice
red   --&gt; Bob
green --&gt;      Alice, Tom

size &lt;= 5 --&gt;      Alice
     5-7  --&gt; Bob, Alice
     7-10 --&gt; Bob, Alice, Tom
     &gt;10  --&gt; Bob, Alice


Range.  I have a hammer; I want your problem to be a nail.  My hammer is a way to index ip-address ranges for very efficient lookup.  It is so efficient that the "size" test becomes a single hit to the table that will find the set of persons that match.  This, in my mind, solves the "range" part of your problem.

Set.  This part is simply to map one color to a collection of persons.

Collection of persions

Plan A:  the datatype SET is limited to 64 persons.  Not acceptable for "tens of thousands".

Plan B:  a BLOB of bits, each bit representing one person.  This has the distinct advantage of AND and OR now become bitwise operations.  Drawback:  bitwise operations only work with SET and BIGINT.  That is, until MySQL 8.0, when it works on BLOB.  Also, I have implemented bitwise operations on `BLOB.  Another issue:  the bit strings may be very sparse?  So, Plan B is a "maybe".

Plan C:  an ordinary table:  (blue,Bob),(blue,Alice),(red,Bob), ...  Plan C may be the best.  Necessarily, the PRIMARY KEY should be (color, person) to take advantage of 'clustering'.

References

Efficient IP address lookup - for your ranges

Unique Users - someday I may publish bitwise manipulations on BLOB here.

Bottom line.  I have given you some food for thought.  And there is a size limit on Answers in this venue.  And this is a bigger task than I am willing to do for free.