Escape characters in format
After trial and error I figured out you can escape characters using double backslash. 

{{ article.postDate.format('j\\&lt;\\s\\u\\p\\&gt;S\\&lt;\\/\\s\\u\\p\\&gt; F Y') | raw }}


Outputs: 14&lt;sup&gt;th&lt;/sup&gt; January 2016

In search of documentation I found a mention of double-escaping regular expressions due to Twig auto-escaping \. At this point I'm guessing this works for most Craft-Twig functions.