Pick an object up and then drop it
Instead of using an empty, you could animate the object itself. To do so, keyframe the 2 following frames: The one just before it is supposed to be affected by the physics (Animated ON), and the following one, when it is supposed to be affected by the physics (Animated OFF).