NOT gate with PNP transistor
It's the mirror image of the NPN solution: emitter to Vcc instead of ground, load resistor between collector and ground instead of between collector and Vcc.  

edit
Like JustJeff says, make sure your input high voltage is high enough. If it is less than Vcc - 0.6V the transistor will be always on, or at least conduct partially, which shows when your LED isn't completely off. You can make the base voltage higher by adding a resistor from base to Vcc. Together with the base resistor it will form a divider. Example: say Vcc = 5V and your input high voltage is 4.2V. That's too low, the transistor will always be off. If you place a resistor between base and Vcc the same value as your input resistor, the base will be at 4.6V and the transistor will be off.