title: Passing around/Serving Large Model Object in Poltly Dash and using it in multiple callbacks
body: <p>I have what I believe is a relatively technical question about dash callbacks and enabling user interaction with a large simulation model.</p>
  <p>Currently I have in my repository an app directory which contains all things plotly <code>dash</code> related and an <code>src/engine</code> directory which contains all things simluation related.</p>
  <p>Within the app I would like now to code some callbacks to allow the user to interact with the simulation models. They can select things like different scenarios, years, quantiles etc. There is a simulation object class that I have coded that creates and object with an API that allows for these types of queries. The object itself it's about 3 GB because it relies on some baseline data (naturally).</p>
  <p>So currently in my <code>src/engine</code> there is a file called <code>simulation.py</code>.</p>
  <pre class="lang-py prettyprint-override"><code># Contains simulation object ...
  
  class Simulator(object):
  
      def __init__(data1, data2, data3):
          ## Stuff
  
      ... more stuff helper functions etc ...
  
      def query(self, q):
          ## Run a simulation for the query q
          return simluation_based_on_q
  </code></pre>
  <p>In my app directory I first need to get a simulator object so I have a <code>models.py</code> file where I can load my multiple models that I need to be interactable with the app.</p>
  <p>So in <code>app/models.py</code> I have this:</p>
  <pre class="lang-py prettyprint-override"><code>import sys
  
  sys.path.append('../src/engine/')
  
  from simulation import Simulator
  import module_io import DataLoader1, DataLoader2, DataLoader3
  
  data1 = DataLoader1.get()
  data2 = DataLoader2.get()
  data3 = DataLoader3.get()
  
  sim_object = Simulator(data1, data2, data3)
  </code></pre>
  <p>Now in my app directory I have multiple callbacks that can interact with the model.</p>
  <p>So in <code>app/controller/callbacks.py</code> I have something like this.</p>
  <pre class="lang-py prettyprint-override"><code>import dash_bootstrap_components as dbc
  from models import sim_object
  from server import app
  
  @app.callback(
      Output('years-dropdown', 'children')
      Input('years-dropdown', 'id'), 
  )
  def years_dropdown_children(id):
      possible_years = simul_object.years
      return [dbc.DropdownMenuItem(year, id=f&quot;{id}-item-{year}&quot;) for years in possible_years]
  
  @app.callback(
      Output('quantiles-dropdown', 'children')
      Input('quantiles-dropdown', 'id'), 
  )
  def quantiles_dropdown_children(id):
      possible_quantiles = simul_object.quantiles
      return [dbc.DropdownMenuItem(quantile, id=f&quot;{id}-item-{year}&quot;) for quantile in possible_quantiles]
  
  @app.callback(
      Output('heatmap-graph', 'figure')
      [Input('dropdown-years', 'value'), 
       Input('dropdown-quantiles', 'value'),
  )
  def heatmap_graph_update(years, quantiles):
      q = [years, quantiles]
      simulation_output = sim_object.query(q)
      ## plot the simulation
  
  ... So on and so forth ...
  </code></pre>
  <p>Please note that the above is pseudo-code (that is quite close to the actual code) to demonstrate the logic, the app itself is a bit more complex and not possible to share all code in a Stackoverflow question.</p>
  <p>The app was working before creating these callbacks and it even worked when I coded the first callback but as I added more callbacks it crashes when I try to run it. This feels a bit sketchy, but I can't quite put my finger on it, I think its close to &quot;good design&quot; but using the object in multiple callbacks doesn't seem right. I'm thinking that I might need &quot;share&quot; the model object between the callbacks somehow using a redis-cache to cache the model object so each callback is access the same exact version and not somehow creating duplicate objects.</p>
  <p><strong>OVERALL QUESTION</strong></p>
  <p><strong>What is the best methodology/practices for sharing large objects (2-3 GB) between files in dash especially when using the same object in callbacks to generate graphs.</strong></p>