7. A non-transitory computer readable storage medium including instructions that, when executed by a processor, cause the processor to perform a method comprising: receiving digital image data captured by a mobile device; performing image recognition analysis on an object within the digital image data to identify the object; receiving sensor data that describes a change in orientation executed b the user device; determining based on the identity of the object, the image analysis, and the sensor data, a gesture associated with the object; querying a semantic associations database based on the determined gesture and the object to determine a secondary meaning associated with the gesture and the object, wherein the secondary meaning is a non-literal meaning associated with the determined identity of the object based on the gesture, wherein the non-literal meaning is inferred from descriptions associated with a plurality of images located on a computer network that depict objects with the determined identity, wherein the descriptions each include one or more references to the non-literal meaning associated with the gesture; in response to the determination of the secondary meaning, requesting augmentation data from a remote service based on the secondary meaning; receiving the augmentation data from the remote service in response to the request, wherein the augmentation data is relevant to the secondary meaning associated with the identified object, and wherein the augmentation data is to be rendered over the identified object in the digital image data when displayed by the mobile device; and transmitting augmentation data to the mobile device that is semantically relevant to the object based on the secondary meaning.
7. A non-transitory computer readable storage medium including instructions that, when executed by a processor, cause the processor to perform a method comprising: receiving digital image data captured by a mobile device; performing image recognition analysis on an object within the digital image data to identify the object; receiving sensor data that describes a change in orientation executed b the user device; determining based on the identity of the object, the image analysis, and the sensor data, a gesture associated with the object; querying a semantic associations database based on the determined gesture and the object to determine a secondary meaning associated with the gesture and the object, wherein the secondary meaning is a non-literal meaning associated with the determined identity of the object based on the gesture, wherein the non-literal meaning is inferred from descriptions associated with a plurality of images located on a computer network that depict objects with the determined identity, wherein the descriptions each include one or more references to the non-literal meaning associated with the gesture; in response to the determination of the secondary meaning, requesting augmentation data from a remote service based on the secondary meaning; receiving the augmentation data from the remote service in response to the request, wherein the augmentation data is relevant to the secondary meaning associated with the identified object, and wherein the augmentation data is to be rendered over the identified object in the digital image data when displayed by the mobile device; and transmitting augmentation data to the mobile device that is semantically relevant to the object based on the secondary meaning. 9. The computer readable storage medium of claim 7 , wherein the querying further comprises: consulting a translation table that associates objects with one or more secondary meanings.