1. A method of categorizing the relationship of individuals from a collection of photos taken in multiple sets of events, comprising using a computer for automatically performing the following steps: a. searching the photo collection to identify individuals in the photo collection, and determining the gender and age range of the identified individuals; b. determining occurrences of identified individuals and co-occurrences of identified individuals; and c. performing an inference task using the identified individuals, their occurrences and co-occurrences, their gender, and their age ranges to determine a type of social relationship relationships between a pair of individuals, wherein the types of social relationships include parent, child, spouse, other relative, friend, child's friend, or acquaintance.