1. A computer-implemented method comprising: identifying an image associated with a user, wherein the image is identified as at least one of captured by a user device associated with the user, stored on the user device associated with the user, and stored in cloud storage associated with the user; for each of the images, determining one or more labels, wherein the one or more labels are based on at least one of metadata and a primary annotation; generating a mapping of the one or more labels to one or more confidence scores, wherein the one or more confidence scores indicate an extent to which the one or more labels apply to corresponding images; and interacting with the user to obtain identifying information that is used to categorize one or more of the images.