Title: Probabilistic Database Summarization for Interactive Data Exploration,
Abstract: We present a probabilistic approach to generate a small, query-able summary
of a dataset for interactive data exploration. Departing from traditional
summarization techniques, we use the Principle of Maximum Entropy to generate a
probabilistic representation of the data that can be used to give approximate
query answers. We develop the theoretical framework and formulation of our
probabilistic representation and show how to use it to answer queries. We then
present solving techniques and give three critical optimizations to improve
preprocessing time and query accuracy. Lastly, we experimentally evaluate our
work using a 5 GB dataset of flights within the United States and a 210 GB
dataset from an astronomy particle simulation. While our current work only
supports linear queries, we show that our technique can successfully answer
queries faster than sampling while introducing, on average, no more error than
sampling and can better distinguish between rare and nonexistent values.