Events calendar & time
Drupal 7?

Use admin/config/regional/date-time/formats and create a new Date/Time format that only outputs the time. Then in your view set the field display to use that format by selecting  "date and time" under "formatter." Now select your newly created format under "choose how users view dates and times."

I think that is the same in D6 as well or at least similar.