What distinct challenges are faced in taxi operations that aren't found in other phases of flight?
Well in air you don't have to worry about kangaroos in your path or a wheel falling off unnoticed, or people beside a taxiway mistakenly indicating you won't hit a golf cart.  Heaven forbid you accidentally retract the landing gear.

Jokes aisde, taxiing can be a major challenge even for experienced pilots. First off, your traditional warning systems don't work.  You can't check your route using a low or mid-range GPS because they don't have airport taxiway information.  You won't get a "terrain!" warning when you're about to head off the end of a taxiway. 

Steering is difficult because the control surfaces are used differently on the ground, especially in a crosswind.  You don't use your ailerons or elevator to control your path; you use them to keep the wings from lifting and to keep you on the ground.  You use the rudder and steerable nose wheel/tail-wheel (if present) to keep you on the right path, with help from the brakes only when necessary.   Those brakes definitely will be necessary in a tail-wheel aircraft with a strong enough crosswind.  If you fail to control the ground path in a tail-wheel aircraft, an unstable and difficult ground loop can develop. 

Did wbeard52 mention low visibility? You can technically land a plane in zero visibility if your plane and the airport are properly equipped, but good luck naviagting the taxiways afterwards. A pilot once remarked that "Taxiing at ORD the first time in low visibility conditions is impossible," and another remarked that he was so confused by low visibility he had to have a follow-me truck guide him. [source]

Taxiing can also be confusing in good weather, especially at night.  Sometimes the taxiway letters jump randomly around the alphabet, sometimes markings are obstructed, sometimes there's construction (you don't see construction much in the air!).   The FAA has an entire handbook appendix devoted to avoiding runway incursions that explains these navigational issues.  

You also have to be vigilant for obstacles around you (like those kangaroos I mentioned earlier).  In a tail-wheel aircraft with a high nose, for example, it's recommended to zig-zag a little so you can see what's right in front of you.   In aircraft like the B777-300 they actually have camera systems to augment visibility.  (see this question and the FAA's Airplane Flying Handbook ).

You can also look up a list of recent taxiing incidents here