Can I drill a small hole in a concrete beam?
As you (the OP) have discovered, small holes in concrete are perfectly acceptable. 

No disrespect to other posters, but likening concrete to glass is inaccurate. Also, hitting chunks of igneous rock is never a problem in my experience. (Hitting rebar is a problem, though. You'll want to be careful when drilling... If you suddenly stop making progress and you've already cleared the dust, then pull back and either try to subtly change the angle or drill a different hole.)