What component of an aircraft is most likely to fail?
Aside from the obligatory "The Pilot" answer, there are some components on an aircraft that will certainly fail at a higher rate than other components.
From the Piston GA side of the equation I can name a few (some of which are also applicable to turbine aircraft).


Incandescent Light Bulbs
A lightbulb is a supremely unreliable device: A delicate bit of wire is suspended in a thin glass envelope and heated until it glows - then subjected to vibration (from the engines, turbulence, etc.) in flight. We use them all over the airplane -- Position lights, landing lights, cockpit instrument lighting, etc. If you think about how often incandescent light bulbs burn out in your house it's probably not surprising that they fail quite often on aircraft. One such failure lead to a famous crash.  


  We still use incandescent lightbulbs because they're cheap: A $0.25 light bulb can be replaced multiple times before its cost even approaches the capital investment in a better solution.
  Modern aircraft are incorporating an increasing number of LEDs to replace incandescent lightbulbs - these solid-state devices have a lower failure rate (and other benefits).

Batteries
Much like your car aircraft rely on batteries for some functions (like starting the engines or the APU). Batteries have a finite service life and will eventually fail to hold a charge and need to be replaced.
Sometimes battery failures can be rather dramatic.  


  We still use batteries because hand-propping aircraft can be dangerous, and getting the APU turbine up to speed without a battery would be nearly impossible. Also if the engine-driven generators fail the battery is a useful source of electricity to help you get to a safe landing site.

Spark Plugs
Piston engines are spark-ignition, and spark plugs have a variety of failure modes (they can become fouled with lead or carbon deposits, the internal resistor can fail, or the electrodes can simply wear out).
These failures are not uncommon: Statistically if you fly piston aircraft long enough you'll probably encounter at least one of them.


  We use spark plugs for obvious reasons: Unless the fuel is ignited it doesn't generate any power. Without power the propeller doesn't turn (at least not very fast), so there's no thrust. No thrust means no flying.

Tires
Tires are another aircraft component with multiple failure modes. Interesting ones include leaks (air leaving the tire at a rate fast enough to be a problem), "catastrophic" failures where the tire explodes or disintegrates while rolling, and "self-protective" failures (where the tire reaches a temperature that melts the fusible plug and causes it to deflate to prevent it from exploding).


  Tires are also used for an obvious reason: Aircraft that constantly belly-land require expensive maintenance (to say nothing of the damage they do to the airport), and the alternatives - skis, pontoons, or flying boats - are limited in the areas from which they can operate.

Air Filters
On piston engines there is normally an intake air filter which is designed to keep dust and grit out of the engine. These filters can fail by becoming clogged (which I suppose isn't really a failure: The filter is doing its job), and certain paper filters can fail by disintegrating and being ingested into the engine (the FAA has issued airworthiness directives on this subject).


  We still use air filters - even the paper kinds that can disintegrate - because ingesting dirt into a piston engine greatly accelerates wear. The required maintenance on the air filter costs less than increased overhaul frequency.