1. A computer-readable medium, comprising software encoded thereon to detect passback events, which software when executed by a computer system, causes the computer system to perform operations comprising a method of: defining a passback direction for a video monitored area; accessing video collected from the video monitored area; comparing a track determined from the collected video to the passback direction to detect whether any passback event occurred, comprising: representing said track as a track vector; representing said passback direction as a passback direction vector; determining a dot product of said track vector and said passback direction vector; comparing said dot product to a threshold to determine whether said track vector is substantially aligned with said passback direction; and when said track vector is substantially aligned with said passback direction, filtering said track vector by at least one of spatial filtering or temporal filtering to determine whether a sufficient number of track vectors are substantially aligned with said passback direction; detecting passback events based on the determination; and initiating an action based on the detected passback events.