In this paper a system is proposed to combine small finger movements with the large scale body movements captured from a motion capture system. The strength of the proposed work over previous research is in the real-time and natural interactions that the virtual hands have with their environment. By being able to conform to physics, the virtual hands feel like virtual extensions of one's own hands. This provides a higher degree of immersion and interactivity when compared to more traditional virtual reality systems.