1. A method for representing image information in a view of a real environment on a mobile device, comprising: capturing an image of the real environment; determining an image pose of the image in a reference coordinate system; receiving, from a server device, a reference image and a reference pose of the reference image from a reference database; matching features of a real object depicted in the image to corresponding features of a real object depicted in the reference image from the reference database; generating an updated image pose based at least in part on the reference pose in response to determining that the features of the real object depicted in the image match the corresponding features of the real object depicted in the reference image; receiving, from the server device, an indication of an object pose of a virtual object, the object pose based on the reference coordinate system; determining an overlay position in the image based on the updated image pose and the object pose; displaying the virtual object overlaid at the determined overlay position in the image on a display device; receiving input requesting manipulation of the virtual object within the image; and sending, to the server device, a request to replace the object pose with an updated object pose based on the updated image pose and the input requesting manipulation of the virtual object.