1. A method for analyzing video data in a security system, the method comprising: comparing a background model to at least one frame of the video data to identify at least one patch of changed pixels between the background model and the at least one frame; generating masks based on the at least one patch of changed pixels that define contours around the at least one patch of changed pixels; and applying the masks to the background model and the at least one frame and calculating similarity scores based on the masks to determine whether the at least one patch of changed pixels is characteristic of an abandoned object or a removed object.