Rig with joined objects
The problem are weights. each part of the mesh gets weights automatically by assigning the armature modifier, but these don't work with intersecting objects like yours. It's almost impossible to assign proper weight to cloth details like this manually.
To solve it, try to use Mesh Deform modifier: 


make a very simple mesh around your model.
add Mesh Deform to your original model, bind it with your mesh.
add bones to your simple mesh.