ffmpeg bgr vs rgb and other similiar pixel formats
While I always guessed that the big / little endian was more a matter of patents rather than performance,


Nope little endian was developed as a performance optimization whern moving to multi byte words. https://en.wikipedia.org/wiki/Endianness#Optimization

Nothing to to with patents. Different representation have different advantages and disadvantage. Hence different situations may use different representations. Codecs suffer from network effect. Nobody wants to use your encoder if nobody uses your decoder. And vice-versa. So codec designers go for maximum compatibility, and often support several formats. But the usually don't support them all, because there are dozens. So they use a combination of the most common, and easiest to include into the design.