Panels vs template performance
I think the thing to keep in mind here is how the site is going to be used.  If users are not going to log into it, it almost doesn't matter in terms of performance because you use Boost.

http://drupal.org/project/boost

You can also use caching features in panels.  I've never really noted a large hit for using panels on a website, as long as the layout is reasonable and the site does not have an excessive amount of modules on it.  Panels exposes a lot of functionality, and you can pry it out of my cold dead hands.

If you've not used Boost before, do yourself a favor and commit to doing so!

NOTE: In your other question you referenced D7.  Boost may not be ready for D7 just yet but you should keep it in mind.