Can a regular rasp be used to enlarge a hole in concrete?
The metal on a wood rasp would be way to soft to make any type of dent in the concrete.  The only thing you will do is destroy the rasp.

You should use a roto hammer with the proper sized bit to widen the hole.