title: Dynamic Streaming, Loading, Unloading and Sharing of Game Assets
body: <p>I'm currently designing a system for handling game assets in a game engine, and I was just looking for some input/discussion on the best way to go about it. My current system is very manual and traditional, I'm looking for something more automatic to replace it with. My goal is to achieve a system that:</p>
  
  <ol>
  <li>Dynamically loads and unloads data as it's needed on the fly, such as while navigating large open worlds.</li>
  <li>Shares data between assets that reuse the same data, ensures data is only unloaded from memory if it no longer has any users.</li>
  <li>Manages quality changes/partial or preview or LOD versions of assets.</li>
  </ol>
  
  <p>So far my new system looks something like this:</p>
  
  <p>My current approach is to abstract the concept of Asset loading and unloading, and simply have code refer to 'using' an Asset, or 'unusing' it, and inside of Assets I keep an integer which tracks how many users there are for the asset. When an Asset reaches 0 users, it's removed from the related dictionary.</p>
  
  <p>I use a specific tailored Manager class as a central control for each type of Asset, such as TextureManager or MeshManager. Each AssetManager has a thread running inside of it that handles streaming Assets from outside of the main thread of the game. The separate AssetStreamerThreads run constantly in the background, blocking when there is no work to do, waiting for Jobs to complete from the main thread's queue. When there is Job to do, they grab it, complete it, then return it to a completed queue for the main thread to receive in it's next update.</p>
  
  <p>Each one of these threads handles decoding the Asset from a file on the hard disk (or theoretically anywhere really, downloading even?) and getting it into memory, ready to be simply uploaded to the GPU in an few OpenGL calls. The data is stored inside of the Job that is passed back and forth.</p>
  
  <p>The Managers keep a dictionary of each Asset that has already been loaded with filepaths equating to references to Asset objects already in memory.</p>
  
  <p>Pseudo Code:</p>
  
  <pre><code>// A simple structure to store job data, Data would be tailored to each type of Asset
  class AssetLoadJob {
      boolean complete = false;
      String filepath;
      Asset targetAsset;
      Data data;
  }
  
  // The Asset Manager
  class AssetManager {
      Dictionary&lt;String,Asset&gt; assets;
      Queue&lt;AssetLoadJob&gt; jobsList;
      Queue&lt;AssetLoadJob&gt; jobsComplete;
  
      // Startup and run the streamer thread.
      public AssetManager {
          AssetStreamerThread streamer = new AssetStreamerThread(this);
          streamer.run();
      }
  
      // To fetch an asset to use. Note, the Asset returned may not be loaded, but will be eventually.
      public Asset use(String filepath) {
          if(assets.exists(filepath)) {
              assets.addUser();
              return assets.get(filepath);
          }
          else {
              Asset newAsset = new Asset();
              jobsList.add(new AssetLoadJob(newAsset, filepath));
          }
      }
  
      // Don't need it anymore? Let the manager know.
      public void unuse(String filepath) {
          assets.get(filepath).removeUser();
          if(assets.get(filepath).getUsers() &lt; 1) {
              assets.get(filepath).unload();
              assets.remove(filepath);
          }
      }
  
      // Called before each update() loop in the game engine.
      public void processJobs() {
          foreach(jobsComplete as finishedJob) {
              finishedJob.targetAsset.receiveData(finishedJob.data);
              jobsComplete.remove(finishedJob);
          }
      }
  
      // Accessed by worker thread
      public AssetLoadJob getJob() {
          return jobsList.remove();
      }
  
      // Accessed by worker thread
      public void returnJob(AssetLoadJob job) {
          jobsComplete.add(job);
      }
  }
  
  // The worker thread which handles loading content.
  class AssetStreamerThread {
  
      AssetManager mgr;
  
      public AssetStreamerThread(AssetManager mgr) {
          this.mgr = mgr;
      }
  
      // The out of main thread loop which runs forever.
      public void run() {
          while(forever) {
              AssetLoadJob job = mgr.getJob(); // Blocking until returns valid job.
              // Load job data.. 
              mgr.returnJob(job);
          }
      }
  }
  
  // An abstract example of an Asset. In practice, this might be instead a Texture, Mesh, Sound object, etc.
  class Asset {
  
      private int users = 0;
      private boolean loaded = false;
  
      // Since we can't access users integer directly, these next two methods control increments/decrements.
      public void addUser() {
          users++;
      }
  
      public void removeUser() {
          users--;
      }
  
      public int getUsers() {
          return users;
      }
  
      // The method to Bind an Asset for rendering, such as Binding a texture before drawing an object with it.
      public void bind() {
          if(loaded) {
              // Use loaded data on GPU
          } else {
              // Use placeholder for missing data or just use 'empty' data. Eg: a checkerboard texture for missing textures or solid black 1px x 1px texture, or whatever. A question mark shape for meshes, or simply nothing at all.
          }
      }
  
      // This method is called by the Manager to give the Asset it's data when it's loaded.
      public void receiveData(data) {
          // Upload data to GPU
          loaded = true;
      }
  
      // Called by Manager, informs the Asset it can release resources.
      public void unload() {
          // Unload data from GPU
          loaded = false;
      }
  }
  </code></pre>
  
  <p>Pros:</p>
  
  <ul>
  <li><p>Game code is very simple with a system like this, models can be loaded by simply calling: ModelManager.use("resources/models/model1.m"); and to release the model from memory at the end, a call to ModelManager.unuse("resources/models/model1.m");</p></li>
  <li><p>The system is multithreaded to avoid stutters in framerate while loading large assets.</p></li>
  <li><p>It is very simple to implement in code.</p></li>
  </ul>
  
  <p>Cons:</p>
  
  <ul>
  <li><p>Forgetting to call 'unuse' on an Asset will cause the Asset's 'users' count to remain permanently stuck above 0, so it would never be unloaded, even if it's not in use. While not a major problem since the Asset remains in the dictionary and won't be loaded twice, for large open worlds too big to fit into memory, this could be an issue. <strong>Is this system faulty by design or should I live with it? Should I add a 'memoryPurge' method to be called every so often between game states?</strong> Not sure how to handle this.</p></li>
  <li><p>Textures and Mesh objects would be either loaded in fully, or not at all. There would be no stages inbetween. I'm not even sure how to implement such a partial load. I would like to be able to load for example, different LOD qualities of an object, and progressively load higher qualities of an object as it comes closer in a scene, such as a distant building. I also would like to be able to control Asset quality in a similar way, such as for example textures. If the player changed the quality setting of textures in the main menu, it would be nice if my Asset system handled automatically increasing or decreasing the quality of Assets loaded into memory. <strong>How should I go about incorporating a concept of 'quality' into my assets?</strong></p></li>
  <li><p>A sudden swarm of jobs to my loading thread could clog it up for a long time. Such as the player teleporting to a new area of the game unexpected, or being flung through the air due to a bug in the physics system, etc. <strong>Is this something I should try to guard against? And if so, should I do that in my gameplay code (eg: max player speed) or in my streaming thread?</strong></p></li>
  <li><p>I saw one person design their texture streaming system so that a 4px x 4px version of every texture was loaded on application startup, and after that the 4px version of a texture was used as a placeholder for any texture that wasn't yet loaded, so that nothing was ever 'missing', just always stored in at worst a very low quality. <strong>Is that worth looking into, and if so, how would you implement such a system?</strong></p>
  
  <p>The first thing which comes to mind on that concept is that loading such tiny bits of data like that from thousands of individual files and going through jpeg decompression for each be too laggy right? <strong>Would a special 'texture_preview.cache' file be a good idea for storing that data and loading it already uncompressed directly into memory for quick loading?</strong></p></li>
  </ul>
  
  <p>Conclusion:</p>
  
  <p>This is the best I can come up with so far but I still have so many unknowns in my head. Am I on the right track at least? I know I've asked a lot of questions, but I'm not looking for an answer to every single question. Any answer that addresses some or most of my queries will be accepted. I'm mostly just looking for new directions to consider to finish off this concept before I dive into implementing it. Or warnings from experienced gurus who have been down these roads before me and know where the dragons are.</p>