Rc switch library transmit using protocol 5
I would expect your sending protocol 5 just fine.

But because protocol 2 and 5 are very similar, your receiver gets confused about which protocol it sees when there's noise or other interference.

This is the definition of the two protocols from the code:

{ 650, {  1, 10 }, {  1,  2 }, {  2,  1 }, false },    // protocol 2
{ 500, {  6, 14 }, {  1,  2 }, {  2,  1 }, false },    // protocol 5


Note the last three parts are the same, which are the pulses for 0, 1, and whether or not the high/lows are inverted. 
The first part is also very similar, which is the pulse length. It's so similar that the difference probably falls within the margin for error the receiver software uses.
Only the second parts differs a bit more, which is the sync bit, it's a short high pulse followed by a longer low pulse. It's easy to mistake the one for the other though when there's noise and such.