Title: Non-parametric Message Important Measure: Storage Code Design and Transmission Planning for Big Data,
Abstract: Storage and transmission in big data are discussed in this paper, where
message importance is taken into account. Similar to Shannon Entropy and Renyi
Entropy, we define non-parametric message important measure (NMIM) as a measure
for the message importance in the scenario of big data, which can characterize
the uncertainty of random events. It is proved that the proposed NMIM can
sufficiently describe two key characters of big data: rare events finding and
large diversities of events. Based on NMIM, we first propose an effective
compressed encoding mode for data storage, and then discuss the channel
transmission over some typical channel models. Numerical simulation results
show that using our proposed strategy occupies less storage space without
losing too much message importance, and there are growth region and saturation
region for the maximum transmission, which contributes to designing of better
practical communication system.