Bake to an object with Solidify?
Use your single faced mesh BUT on the end of the sleeves, or neck or any cloth "thickness" just model just a few faces giving the appearance of thickness "U" shaped on the edge like this (see pictures) that way you will get the sense of thickness without the need to apply a modifier or more subdivision surface unnecessarily.






That way you will still have cloth simulation with a very light weight mesh.
Make the cloth collide with very simple capsule polygon parented directly to the bone (no bone weights), hidden from render (wireframe view only). This way you´ll get a nice cloth simulation without the need of solidify. Play with stiffness. Make some vertex groups to "friction" the cloth more on some points.

Hope this is what you´re looking for. Cheers.