How to mount handrail to drywall over concrete
You can drill holes into the concrete with bits made for concrete. Depending on how dense the concrete is you can sometimes do this with a regular drill. (This takes some patience, it's not like drilling into wood. Don't push too hard or you'll break the bit. Just have patience) If the concrete is too dense, a hammer drill may be necessary. And yes they have concrete anchors. They also have special screws made for concrete that don't require anchors.

Another idea would be to put a railing on the other side. Not a hand rail. A railing that attaches to the floor instead of the wall.