Saving a Python dictionary in external file?
Question: I'm working on a code that is essentially a super basic AI system (basically a
simple Python version of Cleverbot).

As part of the code, I've got a starting dictionary with a couple keys that
have lists as the values. As the file runs, the dictionary is modified - keys
are created and items are added to the associated lists.

So what I want to do is have the dictionary saved as an external file in the
same file folder, so that the program doesn't have to "re-learn" the data each
time I start the file. So it will load it at the start of running the file,
and at the end it will save the new dictionary in the external file. How can I
do this?

Do I have to do this using JSON, and if so, how do I do it? Can I do it using
the built-in json module, or do I need to download JSON? I tried to look up
how to use it but couldn't really find any good explanations.

I have my main file saved in C:/Users/Alex/Dropbox/Coding/AI-Chat/AI-Chat.py

The phraselist is saved in C:/Users/Alex/Dropbox/Coding/AI-Chat/phraselist.py

I'm running Python 2.7 through Canopy.

When I run the code, this is the output:

    
    
    In [1]: %run "C:\Users\Alex\Dropbox\Coding\AI-Chat.py"
      File "C:\Users\Alex\Dropbox\Coding\phraselist.py", line 2
        S'How are you?'
        ^
    SyntaxError: invalid syntax
    

EDIT: I got it now. I had to specify the sys.path to import phrase frome
phraselist.py

Here's the full code I have:

    
    
    ############################################
    ################ HELPER CODE ###############
    ############################################
    import sys
    import random
    import json
    sys.path = ['C:\\Users\\Alex\\Dropbox\\Coding\\AI-Chat'] #needed to specify path
    from phraselist import phrase
    
    
    
    def chooseResponse(prev,resp):
        '''Chooses a response from previously learned responses in phrase[resp]    
        resp: str
        returns str'''
        if len(phrase[resp])==0: #if no known responses, randomly choose new phrase
            key=random.choice(phrase.keys())
            keyPhrase=phrase[key]
            while len(keyPhrase)==0:
                key=random.choice(phrase.keys())
                keyPhrase=phrase[key]
            else:
                return random.choice(keyPhrase)
        else:
            return random.choice(phrase[resp])
    
    def learnPhrase(prev, resp):
        '''prev is previous computer phrase, resp is human response
        learns that resp is good response to prev
        learns that resp is a possible computer phrase, with no known responses
    
        returns None
        '''
        #learn resp is good response to prev
        if prev not in phrase.keys():
            phrase[prev]=[]
            phrase[prev].append(resp)
        else:
            phrase[prev].append(resp) #repeat entries to weight good responses
    
        #learn resp is computer phrase
        if resp not in phrase.keys():
            phrase[resp]=[]
    
    ############################################
    ############## END HELPER CODE #############
    ############################################
    
    def chat():
        '''runs a chat with Alan'''
        keys = phrase.keys()
        vals = phrase.values()
    
        print("My name is Alan.")
        print("I am an Artifical Intelligence Machine.")
        print("As realistic as my responses may seem, you are talking to a machine.")
        print("I learn from my conversations, so I get better every time.")
        print("Please forgive any incorrect punctuation, spelling, and grammar.")
        print("If you want to quit, please type 'QUIT' as your response.")
        resp = raw_input("Hello! ")
    
        prev = "Hello!"
    
        while resp != "QUIT":
            learnPhrase(prev,resp)
            prev = chooseResponse(prev,resp)
            resp = raw_input(prev+' ')
        else:
            with open('phraselist.py','w') as f:
                f.write('phrase = '+json.dumps(phrase))
            print("Goodbye!")
    
    chat()
    

And phraselist.py looks like:

    
    
    phrase = {
        'Hello!':['Hi!'],
        'How are you?':['Not too bad.'],
        'What is your name?':['Alex'],
    }
    


Answer: You can use `pickle` module for that. This module have two methods,

  1. **Pickling(dump)** : Convert Python objects into string representation.
  2. **Unpickling(load)** : Retrieving original objects from stored string representstion. 

<https://docs.python.org/3.3/library/pickle.html> code:

    
    
    >>> import pickle
    >>> l = [1,2,3,4]
    >>> with open("test.txt", "wb") as fp:   #Pickling
    ...   pickle.dump(l, fp)
    ... 
    >>> with open("test.txt", "rb") as fp:   # Unpickling
    ...   b = pickle.load(fp)
    ... 
    >>> b
    [1, 2, 3, 4]
    

* * *

Following is sample code for our problem:

  1. Define phrase file name and use same file name during create/update phrase data and also during get phrase data.
  2. Use exception handling during get phrase data i.e. check if file is present or not on disk by `os.path.isfile(file_path)` method.
  3. As use `dump` and `load` pickle methods to set and get phrase. 

code:

    
    
    import os
    import pickle
    file_path = "/home/vivek/Desktop/stackoverflow/phrase.json"
    
    def setPhrase():
        phrase = {
            'Hello!':['Hi!'],
            'How are you?':['Not too bad.'],
            'What is your name?':['Alex'],
        }
        with open(file_path, "wb") as fp:
            pickle.dump(phrase, fp)
    
        return 
    
    def getPhrase(): 
        if os.path.isfile(file_path):
            with open(file_path, "rb") as fp: 
                phrase = pickle.load(fp)
        else:
            phrase = {}
    
        return phrase
    
    if __name__=="__main__":
        setPhrase()
    
        #- Get values.
        phrase = getPhrase()
        print "phrase:", phrase
    

output:

    
    
    vivek@vivek:~/Desktop/stackoverflow$ python 22.py
    phrase: {'How are you?': ['Not too bad.'], 'What is your name?': ['Alex'], 'Hello!': ['Hi!']}
    

