How to help a yellowed brick with a sticker?
A paste of oxygen bleach slathered on liberally, and then some time in the sun will probably fade the yellow nicely. It should not degrade the plastic (any further).

Your real problem is not re-attaching the label, but removing it without tearing or leaving behind most of the backing.

Depending on the material and glue, you might be able to remove it carefully with water, if the glue can be dissolved in water. If you can get the paper off and let it dry flat, you should have little problem re-gluing it. 

The challenge will be the choice of glue. You need something that isn't necessarily permanent when it cures, but can still stick to plastic. White craft glue (not yellow PVC!) should do the trick. But there are many modern adhesives used for non-permanent applications that can bind plastic and paper. There might even be an archival "spray-mount" style compound that won't yellow over time.

Though, honestly, white craft glue is pretty stable. You can even make a paintable slurry from white craft glue that you can apply with a paintbrush to one surface to make the whole thing easier.

But it will be removing and preserving that labels that I suspect will be the greatest challenge.