Chroma key algorithm with unknown background color
Something self-written should be easy. "Background" probably means that the majority of pixels at the border of your image have that color.

So, just get all the border pixels, put them in an array, convert to hue and then do a histogram. Use the maximum histogram bin as your chroma key color.


Starting with border pixels of that color, erode your image on a hue gradient, i.e. make sure that the hue doesn't change too much from its neighboring pixels, and then add pixels to your background mask until you're done OR
just do classical chroma-keying and convert all pixels of colors "close enough" to that color to background mask.