1. A mobile device for annotating objects using multi-modal natural language inputs, comprising: an interface configured to communicate with a location service to determine location information associated with an object accessible to the mobile device; a message service configured to communicate the location information to a storage device configured to store the location information with the object; one or more input devices configured to receive a multi-modal natural language input, wherein the multi-modal natural language input includes a natural language utterance that annotates the object; a speech recognition engine configured to transcribe the natural language utterance into a textual annotation using a dynamic grammar; an agent architecture configured to search the storage device with one or more semantic attributes extracted from the natural language utterance and retrieve the object from the storage device in response to the extracted semantic attributes matching metadata associated with the location information stored with the object in the storage device; and a processing unit configured to label the object retrieved from the storage device with the textual annotation to post-process the object with the textual annotation, wherein the storage device is further configured to store the textual annotation with the annotated object to post-process the annotated object.