<h2 dir="auto">🚀 Feature</h2>
<p dir="auto">Allow dataloader to be chunkable. Meaning, we can break each epoch into N chunks.</p>
<h2 dir="auto">Motivation</h2>
<p dir="auto">for users who have large training set, the pattern often looks like this</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="def train_model(self)
   while True:
           self.train()
           self.iterations += 1

           if self.iterations % VAL_ITER == 0:
                self.val()"><pre class="notranslate"><code class="notranslate">def train_model(self)
   while True:
           self.train()
           self.iterations += 1

           if self.iterations % VAL_ITER == 0:
                self.val()
</code></pre></div>
<p dir="auto">This allows the user the evaluate many times per epoch. However, this means that there will be 2 active dataloader at a time.</p>
<p dir="auto">A more ideal set up would be, to allow the user to specify the length of each chunk of dataloader. So the user can specify that, he/she wants to evaluate every N number of examples. So the train dataloader will exit before the val dataloader will begin. And when validation finishes, it will initialize a new train dataloader to train the next chunk</p>
<h2 dir="auto">Pitch</h2>
<p dir="auto">Currenlty, it means that there will be 2 active dataloader at a time. Which doubles the RAM usage.</p>
<p dir="auto">Supporting this feature will half the memory usage for many users</p>
<h2 dir="auto"><g-emoji class="g-emoji" alias="question" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/2753.png">❓</g-emoji> Questions and Help</h2>
<p dir="auto">If want to add Bert for pytorch community,  can I put the models and weights of BERT into the repository of <strong>pytorch/text</strong> ?<br>
As <strong>pytorch/vision</strong>  is for Models specific to Computer Vision,  but i can't find any NLP models in   <strong>pytorch/text</strong></p>