Unable to hang up phone call
I'm guessing the Dialer froze.  Try holding the Home button, going into Task Manager, and killing the dialier process.

As Matt mentions in the comments,


  You can do the same thing from the home screen by pressing menu then going to settings > Applications > Manage Application then choose the "phone" or "dialer" app and click "stop"