1. A method of clustering colors of images of a video sequence, comprising: clustering colors of a first image of the video sequence to generate a set of color clusters and to distribute said colors in color clusters of said generated set, and, successively for each image following a preceding image of this sequence, building a pixel mask that samples pixels of said following image that are associated with a color difference above a threshold, wherein said color difference is computed for each pixel of this pixel mask between a color value of the same pixel in said following image and a color value of the same pixel in a compensated image generated by applying a map of motion vectors to said preceding image, wherein said map of motion vectors corresponds to a motion from said preceding image toward said following image, applying said pixel mask to said following image in order to obtain a masked image, and distributing colors of said masked image in color clusters of said generated set, resulting in an updated of the distribution of colors in said color clusters.