title: Flask storing large dataframes across multiple requests
body: <p>I have a Flask web app that uses a large DataFrame ( hundreds of Megs). The DataFrame is used in the app for several different machine learning models. I want to create the DataFrame only once in the application and use it across multiple requests so that the user may build different models based on the same data.  The Flask session is not built for large data, so that is not an option.  I do not want to go back and recreate the DataFrame in case the source of the data is a csv file(yuck).  be</p>
  <p>I have a solution that works, but I cannot find any discussion of this solution in stack overflow. That makes me suspicious that my solution may not be a good design idea.  I have always used the assumption that a well beaten path in software development is a path well chosen.</p>
  <p>My solution is simply to create a data holder class with one class variable:</p>
  <pre><code>class DataHolder:
       dataFrameHolder = None
  </code></pre>
  <p>Now the dataFrameHolder is known across all class instances (like a static variable in Java) since it is stored in memory on the server.</p>
  <p>I can now create the DataFrame once, put it into the DataHolder class:</p>
  <pre><code>import pandas as pd
  from dataholder import DataHolder
  
  result_set = pd.read_sql_query(some_SQL, connection)
  df = pd.DataFrame(result_set, columns=['col1', 'col2',....]
  DataHolder.dataFrameHolder = df
  </code></pre>
  <p>Then access that DataFrame from any code that imports the DataHolder class.  I can then use the stored DataFrame anywhere in the application, including across different requests:</p>
  <pre><code>.
  .
  modelDataFrame = DataHolder.dataFrameHolder
  do_some_model(modelDataFrame)
  .
  .
  </code></pre>
  <p>Is this a bad idea, a good idea, or is there something else that I am not aware of that already solves the problem?</p>