List Categories by 0-9 and A-Z
To group by the first letter you would simply use the slice filter within Craft's group
filter.

For all non letter characters you can now use Craft's replace filter (it does regex since Craft 2.2)
and replace them with whatever you want these entries to be sorted by.

{% set entriesGrouped = entries|group('title|slice(0, 1)|upper|replace("/[^A-Z]/", "0-9")') %}

{% for letter, entriesInGroup in entriesGrouped %}
    {{ letter }}
    {% for entry in entriesInGroup %}
        {{ entry.title }}
    {% endfor %}
{% endfor %}