Items on top of cloth, and dragging the cloth to move those items [blend files attached]
Much like rigid body physics, the cloth modifier in vers,2.77a at least, won't allow the cloth to be animated or shapekeyed whilst the physics cache is active.
The cache's frame range can be set however (physics properties) so we turn that off at frame 140 when the cloth has settled down.
Whilst the cloth physics is still active, two 'cloth shapers' are underneath the table that physically 'lift' the cloth ends.  The cylinder moves to the right between frames 91 &amp; 98, the other end's triangular cube, from frame 98 to 105.
Both lifters are given Collision Physics.  They don't interfere with other collision objects such as the table and are invisible when rendered.  (eye &amp; camera icons in the Outliner)
In order to animate the cloth, we'd need to  &quot;Apply&quot; the cloth modifier at frame 140 when it's settled down and therefore 'freeze' it in it's current shape, allowing keyframed animation.
In this example we don't need to animate the cloth at all.  Instead we duplicate the table and get rid of the copy's inherited physics so it's just scenery.
The original table is made invisible in the Outliner and remains stationary to support the cloth.
At the frames where the cloth is supposed to be pulled to the right, we instead move the duplicate table to the left with the camera parented to it so the cloth appears to be pulled to the right.

As the falling cubes physics are baked we can add keyframes to simulate their inertia.  All three are parented to an empty so they can be moved as one.
Should there be other objects and scenery behind, simply render the tables, cloth etc. out with a transparent background and render that as one clip.  Render the background objects as another and mix them in the VSE.