{'fraction_non_alphanumeric': 0.04779756326148079, 'fraction_numerical': 0.00937207122774133, 'mean_word_length': 4.741935483870968, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A New Model of Rank-Frequency Distribution

We assume that if there is a homogeneous class of musical or linguistic elements set up on the basis of a property that is relevant for the use of music or language, then the exploitation of these elements abides by its own laws in the sense that their frequency of occurrence displays a regularity which can be modelled by a monotone decreasing rank-frequency distribution. This assumption has three consequences: (i) The frequencies of occurrence of members of any linguistic or musical (jointly “textual”) class are not equal, i.e., they do not follow the discrete uniform distribution, (ii) it is a consequence of some generating mechanism, i.e., it is lawlike (cf. Bunge, 1967, p. 16: “Everything abides by laws”), and it must have developed historically in order to fulfil some human needs (cf. Kohler, 1991), (iii) if one succeeds in finding some laws capturing this phenomenon (and there are a number of successful models) then these laws are in turn the best criteria of “correctness”, “naturalness”, “prolificn...