How bad is it to use a molly in a stud?
Chances are you did not drill a 1 1/2" hole for a molly (into a stud, such that you created a notch), so the rules about notching are not really applicable, but this would be the best kind of guide.

The depth for notching a stud depends on whether or not it is load bearing. For a load bearing stud you can notch up to 25% (7/8") depth; for a non-load bearing stud it is 40% (roughly 1 3/8") depth. 

Now, considering how deeply you drilled and the size of the bit should help elucidate the answer. If you drill bit was less than 3/8", then you should be okay. If the drill bit was more like 3/4" and you drilled more than a couple inches deep, then it could be a problem.