Ask HN: Why do you use a data lake instead of a data warehouse?
I&#x27;m trying to better understand why analysts and data scientists sometimes prefer to set up a data lake, which I define as: &quot;a collection of files, in an open-source format like Parquet or Orc, in a blob-store like S3&quot;. I often hear two reasons:<p>- Cost<p>- Supporting data science workloads<p>However, the emergence of data warehouses that separate compute from storage, like Snowflake, BigQuery and Redshift RA3, has drastically reduced the cost advantage for data lakes. And in my experience, data science workloads are generally compute-bound, so in principal you can just execute a SQL query from your data science environment (Spark, Python) against your data warehouse---the overhead of serializing and deserializing your data an extra time should be a tiny % of your overall runtime.<p>I suspect that the reason why some data scientist prefer data lakes is cultural rather than technical---basically, they have an irrational dislike of SQL and RDBMSs. However, I haven&#x27;t done data science work myself in years, so maybe I&#x27;m just not getting it. Are there good reasons for using a data lake rather than Snowflake, BigQuery, or Redshift RA3? Please enlighten me!