Crystals have a pivotal role in the functional design of inorganic materials. Data-driven approaches have attracted considerable interest for accelerating materials design and exploration . In particular, high-throughput screening of materials and prediction of their properties are attractive options. Various machine learning techniques have been used to accelerate screening . Given the hardest challenge of exploring crystals never seen, generative models have been extensively studied for sampling candidates . However, current models frequently generate stuffs that deviate from chemical principles, specifically from the geometric "concepts" such as the global symmetries of crystals and local atomic environments. Although some symmetries have been considered as constraints , global symmetries and local atomic environments have not been integrally embedded in a feature space. Such a geometric-concept aware feature space would not only augment existing models but also help scientists' intuition or deduction with unveiling relationships between the concepts, inductively from the ever-increasing data reported in the world. Foundation models (FMs) have recently emerged on the basis of techniques from natural language processing, computer vision, and any other machine learning communities . They are trained on a large amount of data with multiple modalities of concepts. They have been evolving interactively with large language models (LLMs) , which have also brought a significant impact on materials science . FMs and LLMs often refer the same as natural language is so common for us, which should also be the case for materials science; however, FMs more specifically connect different modalities and are expected to open up ways of conceptual representation. After the studies of contrastive Figure : Intuition behind CLICS. Crystals and geometric concepts are contrasted on their shared feature space to learn the underlying concepts of crystal structures. Here, the crystal on the left is positively paired with the octahedron, the connected octahedra, and the tetragonal symmetry. The structure is negative against the tetrahedron, the connected tetrahedra, and the cubic symmetry, which are positively paired with the crystal on the right. The two crystals will be represented in graphs (Section 3). The local geometric environments (e.g., octahedron and tetrahedron on the top), the space groups (e.g., two placed in the middle), and the connected geometries (e.g., corner-sharing octahedra and corner-sharing tetrahedra on the bottom) will be represented in texts (Section 2, 3). Each double-headed arrow represents a negative pair (repulsion).