title: Python: Local Database for storing and retrieving historical data
body: <p>I'm trying to create some weather models and I want to store and retrieve data on my hard drive.</p>
  <p>Data is in this format:</p>
  <pre><code>{'Date_Time':'2020-07-18 18:16:17','Temp':29.0, 'Humidity':45.3}
  {'Date_Time':'2020-07-18 18:18:17','Temp':28.9, 'Humidity':45.4}
  {'Date_Time':'2020-07-18 18:20:17 ','Temp':28.8, 'Humidity':48.3}
  </code></pre>
  <p>I have new data coming in every day, I have old data from ~5 years ago.
  I would like to periodically merge the data sets and create one large data set to manipulate.</p>
  <p>Things I need:</p>
  <pre><code>1. Check if the date-time pair already exists, else add new data
  2. Change old data values
  3. Add new data values to the database
  4. Must be on a local storage, I have plenty of space.
  </code></pre>
  <p>Things I would like but do not need:</p>
  <pre><code>1. Fastest Read access possible, not so concerned about storage time as that happens in the background mostly.
  2. Something that makes searching for all data from today, last 7 days etc easy to retrieve
  </code></pre>
  <p>Things I have tried:</p>
  <ol>
  <li><p>Appending to a json file</p>
  <p>Works for now but is slow because I have to load the entire data set every time I want to append/modify</p>
  </li>
  <li><p>Appending to a text file</p>
  <p>Easy to store, but hard to modify/check values</p>
  </li>
  <li><p>SQLLite3</p>
  <p>I looked into this and it seemed workable, just wanted to know if there was something better before I just go ahead and do this.</p>
  </li>
  </ol>
  <p>Thank you for your help!</p>