Why are only captains allowed to taxi the aircraft?
Taxiing can sometimes be one of the more difficult tasks required on a flight, and affords one of the few opportunities to run into something.

Historically, in the earlier days of crewed flying, a captain may have done most or all of the flying. A captain might have let the copilot fly some of the tedious cruise portion of the flight, and a nice captain might have even let a copilot land or takeoff. As CRM developed, the concept of both pilots sharing the flying duties was eventually standardized. 

Most larger aircraft are steered in taxi primarily by means of a steering tiller. When installed, a steering tiller is always installed at least on the left side of the cockpit. In the past, aircraft did not offer an option for a second steering tiller on the right side, likely reflecting the norms of the day. Updated aircraft and newer designs commonly include as standard—or offer the option of—a steering tiller on the right side as well. Some companies choose to omit this option as a weight and cost saving measure.

Some companies allow either pilot to taxi the airplane unless the aircraft is not properly equipped to allow this. I do not have personal knowledge of any companies that forbid the first officer from taxiing, but such policies may well still be in place. In many cases it is still not an option since the aircraft is not properly equipped to be taxied from the right seat.