1. A method of tracking objects of a scene, said method comprising: determining two or more tracks which have merged, each track being associated with at least one of the objects and having a corresponding graph structure, each graph structure comprising at least one node representing the corresponding track; creating a new node representing the merged tracks; adding said graph structures as children nodes of the new node to create a merged graph structure; detecting a split between the objects associated with one of the tracks represented by said nodes of the merged graph structure; determining similarity between one or more of said nodes in the merged graph structure and foreground areas corresponding to split objects; selecting one of said nodes in the merged graph structure based on the determined similarity; and creating a new graph structure for tracking the objects in response to the detection of the split, the new graph structure having the selected node at the root of the new graph structure.