Smart mobile devices have fostered new interaction scenarios that demand sophisticated interfaces. The main developers of operating systems for such devices provide APIs for developers to implement their own applications, including different solutions for graphical interfaces, sensor control, and voice interaction. Despite the usefulness of such resources, there are no strategies defined for coupling the multimodal interface with the possibilities that the devices offer to identify and adapt to the user needs, which is particularly important in domains such as Ambient Assisted Living. In this paper, we propose a framework that allows developing context-aware multimodal conversational agents that dynamically incorporate user-specific requirements and preferences as well as characteristics about the interaction environment, in order to improve and personalize the service that is provided. Our proposal integrates the facilities of the Android API in a modular architecture that emphasizes interaction management and context-awareness to build user-adapted, robust and maintainable applications. As a proof of concept, we have used the proposed framework to develop an Android app for older adults suffering from Alzheimer's. The app helps them to preserve their cognitive abilities and enhance their relationship with their environment.