1. A selection device for selecting a nearest neighbor set among a plurality of elements, the nearest neighbor set being a set of elements each having a relation to a predetermined reference element, a degree of the relation meeting a predetermined condition, the selection device comprising: a nearest neighbor set candidate selection unit for selecting, among the plurality of elements, a set of elements determined that the set of elements have stronger relations to the reference element, as each of a plurality of nearest neighbor set candidates, which are a plurality of different candidates for the nearest neighbor set; a nearest neighbor set selection unit for selecting, as the nearest neighbor set, a set of elements having relations, degrees of which meet the predetermined condition, to the reference element, among a union of the plurality of nearest neighbor set candidates; and a hierarchical structure data generation unit which sets the plurality of elements as nodes, generates a plurality of hierarchical structure data having a data structure in which the plurality of nodes are classified into any of a plurality of hierarchies, and generates an edge between a node classified into each of the plurality of hierarchies and another node adjacent to the hierarchy, for each of the plurality of hierarchical structure data, based on a degree of relation between the node and the other node, wherein, for each of the plurality of hierarchical structure data, the nearest neighbor set candidate selection unit selects one nearest neighbor set candidate based on the hierarchical structure data.