Performance and loading times of theme like Zen vs building a theme from scratch?
Although I have used Fusion, instead of Zen, my personal experience is probably applicable.  You can run some comparisons using Zen vs. Stark, which is very stripped down to get an idea how they compare.  There are, of course plenty of other factors that can contr5ibute to the equation. 

Regardless of the performance issue, there is also the development question.  Styling one of these themes can be a challenge especially if you include panels, views, superfish menus, or any other module that may also affect appearance.  I had a full head of hair when I started.  

I found it much easier to use Panels-Everywhere and Panels to bypass the them system and create my own display formats. This removes a lot of inherited CSS and made styling much easier.