Replacing an aggregation system of statistics
Disclosure: I am the product manager of Google Cloud Bigtable.

Is it fair to assume that these statistics relate somehow to a globally-unique identifier (user id, device id, something else)?

If so, you might be able to do everything you need in Bigtable, depending on the specific set of queries you need to issue. For example WePay does something quite similar, and they presented their solution and schema design in Bigtable for flexible aggregation of metrics data.

Since Bigtable is a wide-column NoSQL database, you can create a number of columns for any given key and store raw data in one column (with time series for events), partial aggregations for minutes, days, weeks, months, etc. each in separate columns, and then when you need a particular aggregation over an arbitrary time interval, read the partial aggregations as needed.

You can also do this in two different modes, depending on the trade-offs between cost and time:


if only some of your aggregations are accessed, compute them lazily on first access, and write them to Bigtable for storage and querying
if many of the aggregations are needed, and latency is important even for first access, compute eagerly what you can with Dataflow live, and post-compute what cannot be computed live with a background process


As an additional data point, Fastly recently posted how they moved their historical statistics from MySQL to Bigtable, but I don't have any further information on their schema design; hopefully, they will post a follow-up to the announcement with more technical details of their implementations.

If you'd like to discuss your specific use case in more detail privately, let's chat further.