Why do my rear caliper cable becomes loose after some squeezes in the brake lever?
If you fasten the cable anchor with the quick release at its "tight" position and then it's able to slip into a "loose" position, that sounds like a likely culprit to me. I haven't encountered a misbehaving BR-5800 QR yet so I don't actually know what to tell you there, but I think either the silver part in front comes off and is covering a bolt head that holds the whole thing together, or the whole anchor assembly threads in and out of it. I would investigate that and get the issue you describe taken care of either way. You want those dots to have zero ability to come unaligned unintentionally.

Other possible causes of the cable slackening issue:


The cable is slipping from the anchor bolt.
You've got braided    housing that's bursting through a ferrule,
possibly the hidden one    going into the brake lever. Or it could have been installed without a ferrule. (Coil housings
don't need a ferrule here    but braided/compressionless ones do.)
Someone installed a poorly fitting ferrule at the brake lever end
and it's gradually getting wedged further in.
The cable is in the process of fraying apart    somewhere.
There's a rubber cable doughnut or three caught in a    housing stop
getting gradually squished more and more.
The piece of housing going into the brake lever isn't long enough, and as you squeeze the brake, it's getting worked in further as it gets increasingly kinked where it leaves the tape.