1. A method for generating a three-dimensional shape in three-dimensional space comprising: defining a three-dimensional interaction space in the proximity of a user; visualizing the interaction space with a three-dimensional camera system operable to generate three-dimensional coordinate data corresponding to three-dimensional physical objects within the interaction space; interpreting a three-dimensional physical hand gesture of the user within the interaction space according to pre-determined rules, the three-dimensional physical hand gesture including a movement of a three-dimensional physical object and wherein the interpreting is performed at low latency and includes detecting intended interactions using pattern mining, wherein the pattern mining selects relevant salient features by detecting frequent gesture patterns and discarding irrelevant features of the three-dimensional physical hand gesture, wherein the interpreting further includes applying a classification method to only the selected relevant salient features to obtain an interpretation of the three-dimensional physical hand gesture; generating or manipulating a virtual shape based upon the obtained interpretation of the three-dimensional physical hand gesture, the virtual shape residing virtually within the interaction space; and interactively displaying a representation of the virtual three-dimensional shape during the three-dimensional physical hand gesture.