title: Influxdb bulk insert using influxdb-python
body: <p>I used <code>influxDB-Python</code> to insert a large amount of data read from the <code>Redis-Stream</code>. Because Redis-stream and set maxlen=600 and the data is inserted at a speed of <code>100ms</code>, and I needed to retain all of its data. so I read and transfer it to <strong>influxDB(I don't know what's a better database)</strong>, but using batch inserts only <strong>⌈count/batch_size⌉</strong> pieces of data, both at the end of each batch_size, <strong>appear to be overwritten</strong>. The following code</p>
  <pre><code>import redis
  from apscheduler.schedulers.blocking import BlockingScheduler
  import time
  import datetime
  
  import os
  import struct
  from influxdb import InfluxDBClient
  
  def parse(datas):
      ts,data = datas
      w_json = {
      &quot;measurement&quot;: 'sensor1',
      &quot;fields&quot;: {
          &quot;Value&quot;:data[b'Value'].decode('utf-8')
          &quot;Count&quot;:data[b'Count'].decode('utf-8')
          }
      }
      return w_json
  
  def archived_data(rs,client):
      results= rs.xreadgroup('group1', 'test', {'test1': &quot;&gt;&quot;}, count=600)
      if(len(results)!=0):
          print(&quot;len(results[0][1]) = &quot;,len(results[0][1]))
          datas = list(map(parse,results[0][1]))
          client.write_points(datas,batch_size=300)
          print('insert success')
      else:
          print(&quot;No new data is generated&quot;)
  
  if __name__==&quot;__main__&quot;:
      try:
          rs = redis.Redis(host=&quot;localhost&quot;, port=6379, db=0)
          rs.xgroup_destroy(&quot;test1&quot;, &quot;group1&quot;)
          rs.xgroup_create('test1','group1','0-0')
      except Exception as e:
          print(&quot;error = &quot;,e)
      try:
          client = InfluxDBClient(host=&quot;localhost&quot;, port=8086,database='test')
      except Exception as e:
          print(&quot;error = &quot;, e)
      try:
          sched = BlockingScheduler()
          sched.add_job(test1, 'interval', seconds=60,args=[rs,client])
          sched.start()
      except Exception as e:
          print(e)
  </code></pre>
  <p>The data changes following for the influxDB</p>
  <pre><code>&gt; select count(*) from sensor1;
  name: sensor1
  time count_Count count_Value
  ---- ----------- -----------
  0    6           6
  &gt; select count(*) from sensor1;
  name: sensor1
  time count_Count count_Value
  ---- ----------- -----------
  0    8           8
  
  &gt; select Count from sensor1;
  name: sensor1
  time                Count
  ----                -----
  1594099736722564482 00000310
  1594099737463373188 00000610
  1594099795941527728 00000910
  1594099796752396784 00001193
  1594099854366369551 00001493
  1594099855120826270 00001777
  1594099913596094653 00002077
  1594099914196135122 00002361
  
  </code></pre>
  <p>Why does the data appear to be overwritten, and how can I resolve it to insert all the data at a time?</p>
  <p>I would appreciate it if you could tell me how to solve it?</p>