How to clean a rear derailleur?
Lean the bike slightly against something with the derailer facing out.  Slide a section of newspaper up behind the derailer and large cog and let it drape down over the tire and onto the ground.  (This protects the tire and absorbs most of the solvent.)  

Get an old toothbrush and a spray can of (no shouting now!) WD40 or some other similar solvent.  Alternately spray the derailer (and cluster, if you're so inclined) with WD40 and work it with the toothbrush.  

When it's sufficiently clean (ie, no longer caked with congealed crud) wipe it dry and lubricate with chain oil.  (Since this is usually done in tandem with a chain cleaning, simply oiling the chain will usually oil the derailer sufficiently.)  

You may want to hit it with another shot of oil after a day or two, since the small amount of remaining solvent may prevent the oil from reaching all surfaces at first.