Ambient Intelligence and Ubiquitous Computing are carrying the world to a reality where almost every object interacts with the environment, either via sensors or actuators, and users must learn how to interact with such systems. This paper presents a gesture-based interaction approach to manipulate such objects. We developed a prototype using a leap motion controller as a hand-tracking device, and a Support Vector Machine as a classifier to distinguish between gestures. Our system was evaluated by 12 users with over 10 commands. We also show a review on gesture-based interaction and compare other proposals with ours.