<p dir="auto">Hey there,</p>
<p dir="auto">How does one actually use class_weight on <code class="notranslate">model.fit</code>?</p>
<p dir="auto">I had originally written the following method to do this, but I'm not entirely sure whether it works or not.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="def calculate_class_weights(train_label):
    list = train_label.tolist()

    num_neg = list.count(0)
    num_pos = list.count(1)

    duplicate = num_pos / num_neg

    class_weights={0 : (num_neg * (duplicate)) , 1: num_pos }
    return class_weights"><pre class="notranslate"><span class="pl-k">def</span> <span class="pl-en">calculate_class_weights</span>(<span class="pl-s1">train_label</span>):
    <span class="pl-s1">list</span> <span class="pl-c1">=</span> <span class="pl-s1">train_label</span>.<span class="pl-en">tolist</span>()

    <span class="pl-s1">num_neg</span> <span class="pl-c1">=</span> <span class="pl-s1">list</span>.<span class="pl-en">count</span>(<span class="pl-c1">0</span>)
    <span class="pl-s1">num_pos</span> <span class="pl-c1">=</span> <span class="pl-s1">list</span>.<span class="pl-en">count</span>(<span class="pl-c1">1</span>)

    <span class="pl-s1">duplicate</span> <span class="pl-c1">=</span> <span class="pl-s1">num_pos</span> <span class="pl-c1">/</span> <span class="pl-s1">num_neg</span>

    <span class="pl-s1">class_weights</span><span class="pl-c1">=</span>{<span class="pl-c1">0</span> : (<span class="pl-s1">num_neg</span> <span class="pl-c1">*</span> (<span class="pl-s1">duplicate</span>)) , <span class="pl-c1">1</span>: <span class="pl-s1">num_pos</span> }
    <span class="pl-k">return</span> <span class="pl-s1">class_weights</span></pre></div>
<p dir="auto">This returns a dictionary of...</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{0: 34, 1: 34}"><pre class="notranslate"><code class="notranslate">{0: 34, 1: 34}
</code></pre></div>
<p dir="auto">Does anyone have a working example of how to balance 2 classes using the <code class="notranslate">class_weights</code> method?</p>
<p dir="auto">Thanks,</p>
<p dir="auto">Keiron.</p>
<p dir="auto">I'm working in a <a href="https://gist.github.com/kleysonr/a41f0d72891afec8a49990c8cc24f5e4">new generator</a> and I realized that keras was calling it more times that it was supposed.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Information about dataset:
  Dataset size: 30
  Training size: 21
  Test size: 9
  Classes: 3
  Batch Size: 6
"><pre class="notranslate"><code class="notranslate">Information about dataset:
  Dataset size: 30
  Training size: 21
  Test size: 9
  Classes: 3
  Batch Size: 6

</code></pre></div>
<p dir="auto">Below my fit_generator:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="kbg = KerasBatchGenerator('/home/kleysonr/Downloads/keras-generator/dataset', batch_size=6, imagesize=(100, 100), test_ratio=0.3)

model.fit_generator(kbg.generate(set='train'), 
                    steps_per_epoch=training_steps,
                    epochs=1,
                    verbose=1,
                    callbacks=[tbCallback],
                    validation_data=kbg.generate(set='test'),
                    validation_steps=validation_steps)"><pre class="notranslate"><span class="pl-s1">kbg</span> <span class="pl-c1">=</span> <span class="pl-v">KerasBatchGenerator</span>(<span class="pl-s">'/home/kleysonr/Downloads/keras-generator/dataset'</span>, <span class="pl-s1">batch_size</span><span class="pl-c1">=</span><span class="pl-c1">6</span>, <span class="pl-s1">imagesize</span><span class="pl-c1">=</span>(<span class="pl-c1">100</span>, <span class="pl-c1">100</span>), <span class="pl-s1">test_ratio</span><span class="pl-c1">=</span><span class="pl-c1">0.3</span>)

<span class="pl-s1">model</span>.<span class="pl-en">fit_generator</span>(<span class="pl-s1">kbg</span>.<span class="pl-en">generate</span>(<span class="pl-s1">set</span><span class="pl-c1">=</span><span class="pl-s">'train'</span>), 
                    <span class="pl-s1">steps_per_epoch</span><span class="pl-c1">=</span><span class="pl-s1">training_steps</span>,
                    <span class="pl-s1">epochs</span><span class="pl-c1">=</span><span class="pl-c1">1</span>,
                    <span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-c1">1</span>,
                    <span class="pl-s1">callbacks</span><span class="pl-c1">=</span>[<span class="pl-s1">tbCallback</span>],
                    <span class="pl-s1">validation_data</span><span class="pl-c1">=</span><span class="pl-s1">kbg</span>.<span class="pl-en">generate</span>(<span class="pl-s1">set</span><span class="pl-c1">=</span><span class="pl-s">'test'</span>),
                    <span class="pl-s1">validation_steps</span><span class="pl-c1">=</span><span class="pl-s1">validation_steps</span>)</pre></div>
<p dir="auto">As I set only 1 epoch, I would expect to see the kbg.generate() delivering only 21 images to keras, but instead, was consumed much more, see the logs:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Epoch 1/1
Batch: 0-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 0-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 0-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 0-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 0-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 0-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 1-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 1-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 1-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 1-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 1-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 1-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 2-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 2-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 2-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 2-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 2-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 2-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 3-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 3-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 3-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
Batch: 4-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 4-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 4-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 4-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 4-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 4-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 5-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 5-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 5-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 5-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 5-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 5-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 6-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 6-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 6-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 6-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 6-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 6-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 7-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 7-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 7-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
Batch: 8-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 8-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 8-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 8-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 8-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 8-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 9-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 9-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 9-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 9-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 9-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 9-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 10-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 10-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 10-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 10-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 10-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 10-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 11-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 11-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 11-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
1/4 [======&gt;.......................] - ETA: 3s - loss: 1.1924 - acc: 0.1667Batch: 12-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 12-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 12-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 12-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 12-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 12-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 13-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 13-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
3/4 [=====================&gt;........] - ETA: 0s - loss: 1.2550 - acc: 0.2222Batch: 13-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 13-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 13-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 13-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 14-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 14-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 14-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 14-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 14-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 14-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
4/4 [==============================] - 1s 351ms/step - loss: 1.2378 - acc: 0.1714 - val_loss: 1.0822 - val_acc: 0.3333"><pre class="notranslate"><code class="notranslate">Epoch 1/1
Batch: 0-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 0-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 0-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 0-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 0-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 0-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 1-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 1-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 1-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 1-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 1-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 1-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 2-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 2-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 2-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 2-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 2-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 2-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 3-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 3-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 3-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
Batch: 4-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 4-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 4-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 4-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 4-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 4-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 5-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 5-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 5-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 5-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 5-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 5-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 6-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 6-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 6-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 6-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 6-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 6-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 7-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 7-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 7-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
Batch: 8-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 8-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 8-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 8-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 8-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 8-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 9-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 9-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 9-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 9-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 9-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 9-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 10-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 10-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 10-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 10-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 10-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 10-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 11-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 11-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 11-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
1/4 [======&gt;.......................] - ETA: 3s - loss: 1.1924 - acc: 0.1667Batch: 12-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 12-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 12-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 12-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 12-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 12-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 13-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 13-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
3/4 [=====================&gt;........] - ETA: 0s - loss: 1.2550 - acc: 0.2222Batch: 13-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 13-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 13-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 13-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 14-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 14-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 14-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 14-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 14-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 14-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
4/4 [==============================] - 1s 351ms/step - loss: 1.2378 - acc: 0.1714 - val_loss: 1.0822 - val_acc: 0.3333
</code></pre></div>
<p dir="auto">Even being necessary data for just a single epoch, keras consumed data for almost 4 epochs.</p>
<p dir="auto">I've include the configuration <strong>workers=0</strong> and <strong>use_multiprocessing=False</strong> to the fit_generator()</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="model.fit_generator(kbg.generate(set='train'), 
                    steps_per_epoch=training_steps,
                    epochs=1,
                    verbose=1,
                    callbacks=[tbCallback],
                    validation_data=kbg.generate(set='test'),
                    validation_steps=validation_steps,
                    use_multiprocessing=False,
                    workers=0)"><pre class="notranslate"><code class="notranslate">model.fit_generator(kbg.generate(set='train'), 
                    steps_per_epoch=training_steps,
                    epochs=1,
                    verbose=1,
                    callbacks=[tbCallback],
                    validation_data=kbg.generate(set='test'),
                    validation_steps=validation_steps,
                    use_multiprocessing=False,
                    workers=0)
</code></pre></div>
<p dir="auto">and I got was it was supposed, enough data for a single epoch</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Epoch 1/1
Batch: 0-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-6.jpg
Batch: 0-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 0-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 0-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 0-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-5.jpg
Batch: 0-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-9.jpg
1/4 [======&gt;.......................] - ETA: 3s - loss: 1.1336 - acc: 0.1667
Batch: 1-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-9.jpg
Batch: 1-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 1-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 1-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-4.jpg
Batch: 1-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 1-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
2/4 [==============&gt;...............] - ETA: 1s - loss: 1.1540 - acc: 0.1667
Batch: 2-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 2-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 2-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 2-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 2-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-6.jpg
Batch: 2-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
3/4 [=====================&gt;........] - ETA: 0s - loss: 1.3066 - acc: 0.2222
Batch: 3-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-1.jpg
Batch: 3-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 3-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
1544964579 --train-- New epoch
4/4 [==============================] - 2s 391ms/step - loss: 1.2351 - acc: 0.3238 - val_loss: 1.1089 - val_acc: 0.2222"><pre class="notranslate"><code class="notranslate">Epoch 1/1
Batch: 0-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-6.jpg
Batch: 0-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 0-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 0-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 0-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-5.jpg
Batch: 0-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-9.jpg
1/4 [======&gt;.......................] - ETA: 3s - loss: 1.1336 - acc: 0.1667
Batch: 1-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-9.jpg
Batch: 1-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 1-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 1-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-4.jpg
Batch: 1-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 1-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
2/4 [==============&gt;...............] - ETA: 1s - loss: 1.1540 - acc: 0.1667
Batch: 2-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 2-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 2-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 2-3 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 2-4 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-6.jpg
Batch: 2-5 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
3/4 [=====================&gt;........] - ETA: 0s - loss: 1.3066 - acc: 0.2222
Batch: 3-0 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-1.jpg
Batch: 3-1 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 3-2 &lt;&lt;train&gt;&gt; /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
1544964579 --train-- New epoch
4/4 [==============================] - 2s 391ms/step - loss: 1.2351 - acc: 0.3238 - val_loss: 1.1089 - val_acc: 0.2222
</code></pre></div>
<p dir="auto">I believe this behavior might be related with the <strong>worker=1</strong> working in a async thread and pushing data to keras without any control.</p>
<p dir="auto">Shouldn't this async thread take in account the number of epoch to calculate and limit how much data should be consumed and pushed to keras ? More data that what is needed is a waste of processing and memory consumption.</p>
<p dir="auto">Best Regards.<br>
Kleyson Rios.</p>