If a battery case falls apart is it safe to just glue it back together?
I don't think that gluing the case back together is a risk. The worst risk is in your case that the battery case comes off again and that the batteries are exposed to other damage like shorting the terminals or puncture by a pointed object.

If just gluing the halves back together will do the trick depends on the mechanical stress the case will encounter in the future: Maybe you want to wrap it in a layer of strong tape or use a pair of cable ties around the case for extra "safety". Make also sure, that your repair has to protect the battery from moisture.