Check to see if category is populated?
There is a great tutorial over at Straight Up Craft for handling this (it is however very similar to what you have - it may speed things up a tad though).


  First, we want to grab all of the entry ids from the Section of
  content which we want to display categories for.
  
  {% set entryIds = craft.entries.section('blog').ids() %}
  
  Next, we will use the craft.categories tags relatedTo() method to only
  return categories that have a relationship with one of our entry ids.
  
  {% set categories = craft.categories.relatedTo({ sourceElement:
  entryIds }).groupId(1).find() %}
  
  Once we have our list of categories that are in use, we can loop
  through those categories and display them in our template:

{% for category in categories %}
    &lt;li&gt;&lt;a href="{{ category.url }}"&gt;{{ category.title }}&lt;/a&gt;&lt;/li&gt;
{% endfor %}