{'fraction_non_alphanumeric': 0.014972419227738377, 'fraction_numerical': 0.0007880220646178094, 'mean_word_length': 5.2254901960784315, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
First Elephants then Mice: A Two-stage Robust Estimation Technique for Traffic Matrix in Large Cloud Networks in Presence of Over Dispersion

Traffic Matrix estimation has always caught attention of researchers for better network management and future planning. However, with the advent of cloud services the traffic patterns have become so diversified that they do not follow one probability distribution such as Gaussian, Poisson, Negative Binomial etc that makes it difficult to model them. Further complicating matters is the fact that the elements of the traffic matrix may exhibit over-dispersion, a trait where variation in traffic volumes surpasses the mean value has become increasingly. We find that over dispersion is a more severe problem with mice flows. Thus, we formulate a two step optimization approach in which elephant flows which are more closer to Gaussian approximation are estimate in first step with reasonable accuracy with more estimation error for over-dispersed mice flows. The solution for over-dispersed mice flows is refined through a second bounded-value optimization step by adding an additional constraint. Experimental results show that prediction can be improved by up to 4 orders of magnitude for the originally ill-estimated flows.