Why was a flight delayed because of a bird in the cockpit?
Having a small bird in a cockpit, or anywhere on board, would not be a problem during a normal flight because a normal flight should have a wide safety margin.

However, a situation can change extremely rapidly from safe-and-normal to heavy workload to full-emergency due to any number of factors, including weather, mechanical issues, airspace/congestion, etc.   In a high stress situation, a small bird could be the proverbial tipping point.  So I see and understand the decision to turn back.

It is remotely possible that a small bird could get somewhere it shouldn't, like behind a panel or near electrical lines and cause a short or other serious problem.  Its not terribly likely, but with 100+ lives at stake, why would you risk it?

As for why they didn't just kill it:  Killing it involves catching it, and catching a small bird while still flying a plane is not easy!!