User can input multiple terms, results can tagged as some or all of those terms, but not other terms
So if I understand this correctly, you only want the nodes that have all the checked values (and not a subset). So if I checked only blue, only node 4 will be returned and not 1 and 3? 

I think you'd be best of to write a custom filter, based on the 'is one of' filter. With that you can select nodes that have the terms attached and than drill down the filter.