Can I use Try_Cast to give me a query result column with different datatypes?
All output of the case expression must be of the same datatype.

If maintaining the XML datatype for the valid XML data is worth a little manipulation of the invalid data, then one workaround could be to wrap the invalid data up like a declaration before casting it to XML:

cast('&lt;?invalid ' + RunLog + ' ?&gt;' as xml)


Also, you shouldn't need to use both try_cast and cast in the same expression.

coalesce(try_cast(RunLog as xml),cast('&lt;?invalid ' + RunLog + ' ?&gt;' as xml))