Rubber cloth preset looks wrinkled on character mesh
One thing is - less vertices = more solid.
I would say your topology in that place self collides. Current self-collision distance cause overlapping a space of several faces in their Normal direction.
Try to lower Physics Properties &gt; Cloth &gt; Collision &gt; Self Collisions &gt; Distance

Default Sphere with 0.004

... but lower value probably make it easier to penetrate.
Or you can try to use a Vertex Group (use vertices without &quot;frame&quot; part so it would be excluded from self-collision), but that is probably that you don't want to ... it defence own situation.
BTW There isn't a reason for Shape &gt; Pin Group, when set zero - pinning doesn't take any effect.