Twig Filters and Text Format Filters
I ended up solving this by running the field through the render filter first, then my custom filter. The text format filters are applied at the render stage, so now the custom filter runs after them. The finished order is:

{{ field | render | custom_filter }}