How important is the channel order in deep-learning computer vision tasks?
Task dependent but could be important.

Lets find some arbitrary representation that separates luminance from chrominance (represented as a color 2D vector)? That way, you are detected bright objects independent of color, and colorful objects less dependent on brightness. Obviously under assumption that you are color agnostic. Check this paper where they applied exactly this augmentation