1. A method to identify an unknown object extracted from a video stream, the method comprising: detecting the unknown object in a video frame, the video frame being one of a plurality of video frames from the video stream; receiving a plurality of inputs, each input in the plurality of inputs being a probable identity of the unknown object in the video frame; associating each input with the unknown object; associating a relative weight with each input, the relative weight being a likelihood of an input to match the unknown object; obtaining values of a distance from each probable identity associated with the unknown object to each reference object; obtaining a joint distance between every unknown object and each reference object, the joint distance being a weighted transformation of distances from each probable identity associated with the unknown object to reference object; comparing the joint distances or input modality distances from the unknown object to each reference object; and identifying the unknown object based on the comparing.