1. A method of segmenting a compressed video containing a plurality of frames into a multi-level hierarchy of video segments, said method comprising the steps of: determining differences between the size of successive compressed frames in media time; and grouping said frames to create a hierarchy of clusters of successive frames as video segments using a clustering process, wherein said clustering process uses the size differences to determine boundaries between clusters at each level of said hierarchy.