How important is the channel order in deep-learning computer vision tasks? I stumbled across this question while working with OpenCV, which stores color images in BGR order in memory, while most other libraries I know of use RGB order.

How important is this difference? Would I be able to use BGR images for an RGB-trained network? Obviously, a fire truck is red in most regions, and not blue. But don't CNNs look for texture rather than simple colors?

Even assuming there is a difference - could it still make sense to apply channel permutations as a kind of data augmentation during training? This would spread the spectrum of textures from each original channel to both others, again, the assumption being that colors are not all that important.

I would be curious to know if research like this already been done - I would be surprised if it had not.
Task dependent but could be important.

Lets find some arbitrary representation that separates luminance from chrominance (represented as a color 2D vector)? That way, you are detected bright objects independent of color, and colorful objects less dependent on brightness. Obviously under assumption that you are color agnostic. Check this paper where they applied exactly this augmentation