Display entries related to Category Field
You want to use the relatedTo filter. See the docs here: https://craftcms.com/support/entries-related-to-category where category is your last URL segment (eg red).

Just a quick addition... if your category URL for the category group matches what you've used above mysite/colors/red and you are using the category template for that group then you will have a category variable already to use in the relatedTo filter.