1. An interface system for providing a low-latency interaction in a virtual environment, the interface system comprising: an interface program executed by a processor of a computing device, the interface program configured to: receive an initial image of a back of a user's hand and corresponding initial depth information that represent an initial actual position of the hand; using the initial depth information, project the initial image into the virtual environment to an initial virtual position, the initial virtual position corresponding to the initial actual position; provide a segmented version of the initial image to a display device for display in the virtual environment at the initial virtual position; receive a subsequent image of the back of the user's hand and corresponding subsequent depth information that represent a subsequent actual position of the hand that is different from the initial actual position; using the subsequent depth information, project the subsequent image into the virtual environment to a subsequent virtual position, the subsequent virtual position corresponding to the subsequent actual position; provide a segmented version of the subsequent image to the display device for display in the virtual environment at the subsequent virtual position; and detect a collision between a three-dimensional representation of the hand and a virtual object in the virtual environment or a physical object.