In this review, we focused on the major classes of fast ion conductors, such as layered oxides, polyhedral connecting frameworks, and cluster anion-type materials, and a comprehensive exploration of their structural and dynamical properties, mechanisms governing ion transportation, and strategies for enhancing ionic conductivity has been undertaken. The pivotal roles played by experimental and theoretical approaches in unraveling the atomistic insights of these materials have been highlighted. This review concluded that fast ion conduction is a complex phenomenon that depends on various factors, such as structural bottlenecks, framework connectivity, carrier concentration, anion rotation, and cation-cation correlation. Based on a systematic understanding of these factors, we suggest some general principles and guidelines for designing and optimizing fast ion conductors.