In RegExr, why is regex OR in JavaScript sometimes not commutative?
It is not a bug; it is rather the JavaScript RegEx engine tries to match as much of the first of the two expressions combined by the "|" operator as it can. Thus, instead of matching a word like "bbb" with as many "b"s as possible, it matches the pattern a* against the empty word (implicitly) at the beginning of "bbb". Granted, the RegExr interface does not highlight this properly, but that is more of a missing feature than a bug.

As mentioned in the comments, you should also be aware the engine tries to find (possibly) strings in the input text which match the pattern specified by the regex; the matches are what are highlighted by the RegExr interface. This is a different use of regexes than in formal language theory, in which regexes are used to specify formal languages, and in which an input (word) as a whole either matches the regex or not.

Caveat: If you are trying regexes verbatim from a formal languages class, you may run into trouble when trying to specify the empty word; the character ε is not interpreted as the empty word but as the Greek lowercase epsilon character instead. This question on SE contains some ways of correctly specifying (only) the empty word.

Warning number 2: If you write L(a*|b*) verbatim for your expression there, it is interpreted as a capital $L$ followed by the pattern (a*|b*), not as the language corresponding to the pattern (a*|b*)!