how to process file one by one in python
<p>I have many many files in a folder, and I want to process them one after another, I need to have a global dictionary to record the user identifier and the flowcount, but if my code is like this, when the second or third file is processed, the user_dict for last file will lose.</p>

<p>Because if an user id in the second file is as same as in the first file, then if should be assigned the same flowcount instead of a new one, how can I make one dictionary keep growing when open files one by one? </p>

<pre><code> for line in fd.readlines():
    obj = json.loads(line)
    user = obj["host_dst"]["addr"] + '_' + str(obj["host_dst"]["port"])
    if user not in user_dict:
       user_dict[user] = []
    user_dict[user].append(obj["params"]["flowcount"])
</code></pre>

<p>Since the size of each file is very big, I merged them all into one file, then ran the script to process, the computer will kill the process after a while, I have to process them one by one instead</p>
