If I plug in a USB wall adapter but don't have a device hooked up, will it use power?
The phrase you are looking for is "Vampire Power." Most phone chargers will have it, but will be minimal, like a few cents a year per cell charger in energy usage. Computer power supplies will be worse. And anything that is on a standby mode, like sleeping computers, TVs, game systems (like the PS2/3, Xbox 360/One, Wii), microwaves, etc. will consume power as well, because while the main functions are off, other functions like time-keeping or waiting for a remote to turn them on still consumes power.

Some cell chargers have been designed to combat vampire power usage, and will have a ton of labels on it telling you such. One of the simplest has a modified usb port, with a spring switch that physically disconnects the power to the regulating supply when there is no usb cable plugged in. Essentially, a power on/off switch.

Wiki has an entire article here: http://en.wikipedia.org/wiki/Standby_power