How do I create a lower third for a music video in After Effects?
This is really just a typical title.  You can prepare it either in Photoshop, Illustrator or After Effects directly.  It can also be created in Premiere with a little work.

At their most basic, an MTV style song information could just be text that you put in a layer that you fade in and fade out of the video.  Alternately, if you want a more complex one, you can use vector objects and animate them using keyframes in After Effects.

You would want to start with a vector graphic from Illustrator or a raster image with a transparent background from Photoshop.  Each keyframe will store the position and rotation of the image you want and the underlying video will be visible wherever it is transparent.

The exact way to animate it depends on what kind of animation you want it to have, however there are a number of keyframe animation questions relating to After Effects that you can take a look at for some general ideas.