{'fraction_non_alphanumeric': 0.01759788825340959, 'fraction_numerical': 0.0008798944126704796, 'mean_word_length': 5.299168975069252, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Sparse and robust kernel methods

Statistical inference inescapably involves dealing with uncertainty, whether “out there” in the real world, or in the parameters of a hypothetical model. A powerful framework built upon the foundation of Bayes’ theorem provides a set of mathematically sound rules for manipulating a probabilistic representation of belief in the light of new evidence. The Gaussian process (GP) is a widely extensible class of models to which all the mechanics of Bayesian inference may be applied: uncertainties are propagated into predictions for “error bars”, and the evidence framework can be employed as a principled method for learning parameters higher in the model hierarchy. This thesis explores methods that allow the wider application of GPs by addressing issues of complexity and robustness. Computationally, the GP becomes unwieldy for modelling relationships between more than a few hundred points due to the cubic scaling of inference and quadratic scaling of prediction times. A generic algorithm is presented, applicable to any sampling distribution, that reduces these costs to O(NM2) and O(M2) for N data points, using a small “active set” of size M . Supervised dimensionality reduction is investigated as a possible source of further advantage, and the model is evaluated for the common task of binary classification. For the convenience of a fully tractable GP inference problem, it is often assumed that data have been corrupted with independent and identically distributed Gaussian noise. In domains of heavy-tailed or heteroscedastic corruptions this assumption is inappropriate. A new noise model is presented in which fluctuations in sample variance are achieved by using a second GP to partition softly between two noise regimes. This design permits an efficient deterministic inference, avoiding the need for slow stochastic sampling of the posterior. The faithfulness of the result using an approximate posterior is compared to that returned by Monte Carlo integration. The model is further shown to come from a certain class of GP mixtures; applications of the design to classification and more general regression and mixture modelling tasks are discussed, in each of which the benefit of deterministic inference is retained.