1. A system for performing an action in a mixed media environment having a first type of medium and a second type of medium, the system comprising: a capture device for capturing an image of at least a portion of a document in the first type of medium; a media storage for storing a plurality of mixed media reality document data structures, each mixed media reality document data structure comprising an action, a representation of a corresponding document in the first type of medium, a location of data within the representation that matches the image and an index linking the action with the representation; a recognition unit for identifying a mixed media reality document data structure from the plurality of mixed media reality document data structures that has a representation that matches the image and for outputting the location of data within the representation that matches the image; and a server coupled for communication with the capture device, the media storage and the recognition unit, the server including an action processor for receiving an action of the identified mixed media reality document data structure, and performing the action of the identified mixed media reality document data structure based on the location of data within the representation that matches the image.