1. A physical activity surface visualization system comprising: a computing device located proximate to a physical activity surface, the computing device having one or more processors; a video capture device coupled for communication with the computing device, the video capture device being adapted to capture a video stream that includes a physical activity scene of the physical activity surface and one or more physical interface objects placed in the physical activity scene and physically interactable with by a user; and a detector executable by the one or more processors to process the video stream to detect the one or more physical interface objects included in the physical activity scene, to identify the one or more physical interface objects that are detectable, to generate one or more events describing the one or more physical interface objects, and to provide the one or more events to an activity application configured to render virtual information on the computing device based on the one or more events, the detector being coupled to the video capture device to receive the video stream.