Tools to preprocess a big data for dashboards? I have a complex dataset with more than 16M rows coming from pharmaceutical industry. Regarding the data, it is saved in a sql server with multiple (more than 400) relational tables. Data got several levels of hierachies like province, city, postal code, person, and antigens measures, etc. I would like to create many dashboards in order to observe the changes &amp; trends happening. I can use Pentaho, R (shiny) or Tableau for this purpose. But the problem is data is so huge, and it take so long to process it with dashboard softwares. I have a choice of making cube and connect it to dashboard. My question here is whether there are  any other solutions that I can use instead of making a cube? I don't want to go through the hassle of making &amp; maintaining a cube. I would like to use a software where I specify relationships between tables, so the aggregation/amalgamation happens smoothly and output processed tables that can connect  to dashboards. I hear Alteryx is one software that can do it for you (I haven't tried it myself, and it is an expensive one!). I understand this task needs two or more softwares/tools. Please share your input &amp; experience. Please mention what tools do you use, size of your data, and how fast/efficient is the entire system, and other necessary details.
We have dashboards that show information about some processes, which have billions of rows in the database. It's not queried directly though, but instead from pre-aggregated data.

We have automated scripts running in database that populate the aggregated data tables specifically for dashboards. In most extreme cases the raw data is so large and comes in at such a high pace that there's a two-tier aggregation in place. First tier will de-normalize the data and 2nd tier will do the actual sum/count type of aggregation.

So you don't need two tools since you can do aggregation purely in database.

edit (answering question from comment): Our dashboards are in Tableau. Data is all in PostgreSQL databases. Automation is done with unix cronjob executing a database function. The function in turn queries which tasks it has to run and runs them. There is a few moving pieces to the whole system, but the architecture isn't too complicated. Write down your ideas and go talk to some architects / engineers - they will know the best way to approach this.