Silencing a rattling u-lock
Anything that will "bunch up" as Mathew suggests should work.  You can slip a piece of tubing over the end of the U, or some O rings, or wrap with electrical tape.  The object is not to have material actually go into the hole where the U fits, but to "bunch up" at the hole.

Otherwise, you could probably run a bungie from the loose piece to some frame member to keep it in tension.