1. A method for simulating interactions using an augmented reality device, comprising: capturing a visual scene for display, wherein the visual scene includes a first physical object and wherein the visual scene is captured using one or more camera devices; simulating, by operation of one or more computer processors of the augmented reality device, a dynamic interaction between the first physical object and one or more virtual objects; and rendering a sequence of frames for display that depict the simulated dynamic interaction the first physical object and the one or more virtual objects.