Drilled into studs, light flickered momentarily but everything else seems fine now - should I be worried?
So you stopped drilling and it got better. Of course, if you put in a screw it might get worse in a hurry. 

Best bet is that you nicked the insulation on a wire or wires - either from drilling too deep, determinedly drilling though something that was supposed to prevent you (or warn you by being difficult to drill, compared to wood anyway) from drilling into it, or because it was installed improperly (too close to the surface, without steel protector plates.) 

As such it would be a very good idea to turn off the breaker for the affected circuit, open up the wall, have a look, and call an electrician if you are not comfortable fixing electrical things yourself.