Output tags in the order they were added to an entry
I managed to solve this by defining the tags as a ElementCriteriaModel. Instead of using the code above I used the following:

{% set tags = entry.tagsFieldHandle %}

{% if tags | length %}
   {% for tag in tags %}
      &lt;li&gt;{{ tag }}&lt;/li&gt;
   {% endfor %}
{% endif %}


Explained further in the Tag section in the docs. {% set tags = entry.tagsFieldHandle %} is a shortcut for:

{% set tags = craft.tags({
   relatedTo: { sourceElement: entry, field: "tagsFieldHandle" },
   order:     "sortOrder",
   limit:     null
}) %}


This meant I didn't need to define the entry relation as the model does that be default and it also sorted the tags as they were defined in the entry with the "sortOrder" (which is also default).