Human granulation intelligence means that people can observe and analyze the same problem from various granulation points of view, which generally acknowledge an essential feature of human intelligence. Each granulation view can generate a granular structure through dividing a cognitive target into some meaningful information granules. This means that a large number of granular structures can be generated from the cognitive target. However, people can group these granular structures and select some representative ones for problem solving. This leads to an interesting research topic: how to efficiently and effectively group a family of granular structures. To address this issue, we first introduce a granular structure distance to measure the difference between two granular structures within a unified knowledge representation. Then, we propose a framework for grouping granular structures, called GGS algorithm, which is used to efficiently partition them. Moreover, two indices denoted as DIS and APD are also designed for evaluating the performance of a grouping result of granular structures. Finally, experiments carried out for nine data sets show that the GGS algorithm comes as a sound solution from perspectives of its convergence, effectiveness and scalability. In this way we have proposed and experimented with the general framework for discovering the structure inherent in granular structures, which can be afterwards used to simulate intelligent behavior of human's abilities of granular structure selection. (C) 2016 Elsevier Inc. All rights reserved.