title: Create django model instances from dataframe without server timeout
body: <p>I am uploading a zipped file that contains a single CSV file. I then unzip it on the server and load it into a dataframe. Then I am creating django objects from it.</p>
  <p>This works fine until my dataframe becomes too large. When the dataset is getting too large the django server shuts down. I am assuming that this happens because every iteration in my loop increases memory and when there is no memory space left it shuts down. I am working with big datasets and want to make my code work no matter how big the dataframe.</p>
  <p>Imagine I have a df like this:</p>
  <pre><code>                       cola   fanta   sprite   libella
  2018-01-01 00:00:00    0        12      12       34
  2018-01-01 01:00:00    1        34      23       23 
  2018-01-01 02:00:00    2         4       2        4 
  2018-01-01 03:00:00    3        24       2        4 
  2018-01-01 04:00:00    4         2       2        5 
  </code></pre>
  <p>Imagine that the columns could be up to 1000 brands and the rows could be more than half a Million rows. Further imagine I have a model that saves this data in a JSONB field. Thus every column is a django object, the column name being the name. The time-stamp combined with the data in the column is the JSONB field.</p>
  <p>e.g.: <code>name=fanta, json_data={ &quot;2018-01-01 00:00:00&quot;: 12, &quot;2018-01-01 01:00:0&quot;: 34 }</code></p>
  <p>My code to unzip, load to df and then create a django instance is this:</p>
  <pre><code>        df = pd.read_csv(
              file_decompressed.open(name_file),
              index_col=0,
          )
          for column_name in df:
              Drink.objects.create(
                  name=column_name,
                  json_data=df[column_name].to_dict(),
              )
  </code></pre>
  <p>As I said, this works, but my loop breaks after having created about 15 elements. Searching the internet I found that <code>bulk_create</code> could make this more efficient. But I have custom signals implemented, so this is not really a solution. I also thought about using <code>to_sql</code>, but since I have to restructure the data of the dataframe I don't think this will work. Maybe not using pandas at all? Maybe chunking the DF somehow?</p>
  <p>I am looking for a solution that works independently of the number of columns. Number of rows is maximum of half a million rows. I also tried a while-loop but same problem occurs.</p>
  <p>Any ideas how I can make this work? Help is very much appreciated.</p>
  <p>My dummy model could be as simple as:</p>
  <pre><code>class Juice(models.Model):
     name = CharField(...)
     json_data = JSONField(...)
  </code></pre>