Blender halftone pattern filter
By inverting the RGB channels before splitting them I was able to create a working setup that is also easier to understand.



Here is the new node setup:



There are a few things that can probably be tweaked and improved, but in essence the problem is solved.