1. A method for encoding multiple view video streams on a video stream encoding system, the method comprising the steps of: identifying, with a segment analyzer, a first plurality of identified image segments within a first image of a first plurality of images, the first plurality of images having been captured at substantially a first time and capturing a first at least partially common view of a first scene; identifying, with the segment analyzer, a second plurality of identified image segments within a second image of the first plurality of images; identifying, with the segment analyzer within the first plurality of identified image segments and within the second plurality of identified image segments, a first set of corresponding image segments, wherein each image segment within the first set of corresponding image segments is a respective image segment that correspond to a respective common object within the first scene, the respective common object having a respective image segment that is present in both the first image and the second image; creating, with a graph generator, a first graph data structure for the first image, the first data graph structure comprising a first plurality of nodes with node in the first plurality of nodes being associated with a respective image segment of the first image that is within the first set of corresponding image segments; and specifying, with the graph generator, at least one link between a first node and a second node within the first plurality of nodes within the first graph data structure, each of the at least one link indicating a relationship between a first image segment and a second image segment, wherein the first image segment is associated with the first node and the second image segment is associated with the second node.