1. A computer-readable storage device having stored thereon computer executable instructions for processing depth information of a scene, the computer executable instructions comprising instructions for: receiving a depth image of the scene, wherein the depth image comprises one or more objects; comparing a value associated with voxels in one or more objects in the depth image with a corresponding value of a reference voxel; identifying voxels as moving when a difference between the value and the corresponding value exceeds a motion threshold; identifying other voxels as non-moving; calculating a foreground score based on a percentage of moving voxels associated with one object; and identifying the one object as moving when the foreground score exceeds a score threshold.