1. A multi-modal video data stream segmentation system, comprising: at least one processor; and memory containing a video segmentation application; wherein the video segmentation application configures at least one processor to perform a multi-modal segmentation of a video data stream including a sequence of frames of video, at least one audio track time synchronized with the sequence of frames of video, and closed caption textual data by: identifying visual segmentation cues within the sequence of frames of video; identifying audio segmentation cues within the at least one time synchronized audio track; performing automatic speech recognition on an audio track from the at least one audio track to generate audio track textual data that is time synchronized to the sequence of frames of video; identifying textual segmentation cues identified from the closed caption textual data; matching at least a portion of the closed caption textual data with the audio track textual data and time synchronizing the closed caption textual data to the sequence of frames of video data based upon the time synchronization of the matching audio track textual data; fuse the visual segmentation cues, the audio segmentation cues, and the textual segmentation cues to form a stream of segmentation cues time synchronized with the sequence of frames of video; and identify segmentation boundaries between frames of video within the sequence of frames of video using at least one classifier based upon the stream of segmentation cues.