Title: Big enterprise registration data imputation: Supporting spatiotemporal analysis of industries in China,
Abstract: Big, fine-grained enterprise registration data that includes time and
location information enables us to quantitatively analyze, visualize, and
understand the patterns of industries at multiple scales across time and space.
However, data quality issues like incompleteness and ambiguity, hinder such
analysis and application. These issues become more challenging when the volume
of data is immense and constantly growing. High Performance Computing (HPC)
frameworks can tackle big data computational issues, but few studies have
systematically investigated imputation methods for enterprise registration data
in this type of computing environment. In this paper, we propose a big data
imputation workflow based on Apache Spark as well as a bare-metal computing
cluster, to impute enterprise registration data. We integrated external data
sources, employed Natural Language Processing (NLP), and compared several
machine-learning methods to address incompleteness and ambiguity problems found
in enterprise registration data. Experimental results illustrate the
feasibility, efficiency, and scalability of the proposed HPC-based imputation
framework, which also provides a reference for other big georeferenced text
data processing. Using these imputation results, we visualize and briefly
discuss the spatiotemporal distribution of industries in China, demonstrating
the potential applications of such data when quality issues are resolved.