namespace on python pickle
Question: I got an error when I use pickle with unittest.

I wrote 3 program files:

  1. for a class to be pickled,
  2. for a class which use class in #1,
  3. unittest for testing class in #2.

and the real codes are as follows respectively.

#1. ClassToPickle.py

    
    
    import pickle
    class ClassToPickle(object):
        def __init__(self, x):
            self.x = x
    if __name__=="__main__":
        p = ClassToPickle(10)
        pickle.dump(p, open('10.pickle', 'w'))
    

#2. SomeClass.py

    
    
    from ClassToPickle import ClassToPickle
    import pickle
    
    class SomeClass(object):
        def __init__(self):
            self.pickle = pickle.load(open("10.pickle", 'r'))
            self.x = self.pickle.x
            print self.x
    
    if __name__ == "__main__":
        SomeClass()
    

#3. SomeClassTest.py

    
    
    import unittest
    from SomeClass import SomeClass
    from ClassToPickle import ClassToPickle # REQUIRED_LINE
    
    class SomeClassTest(unittest.TestCase):
        def testA(self):
            sc = SomeClass()
            self.assertEqual(sc.x, 10)
    
    def main():
        unittest.main()
    
    if __name__ == "__main__":
        main()
    

I ran #1 program first to make pickle file.  
And then, when I run program file #2 alone (i.e. enter "python SomeClass.py"),
it works.  
And, when I run program #3 alone (i.e. enter "python SomeClassTest.py"), it
also works.  

However, when I run program #3 as "unit-test" in eclipse+pydev, it returns an
error message below.

> ======================================================================  
>  ERROR: testA (SomeClassTest.SomeClassTest)  
>  \----------------------------------------------------------------------  
>  Traceback (most recent call last):  
> $ File "/home/tmp/pickle_problem/SomeClassTest.py", line 9, in testA  
>  sc = SomeClass()  
> $ File "/home/tmp/pickle_problem/SomeClass.py", line 8, in **init**  
>  self.pickle = pickle.load(open("10.pickle", 'r'))  
> $ File "/usr/lib/python2.7/pickle.py", line 1378, in load  
>  return Unpickler(file).load()  
> $ File "/usr/lib/python2.7/pickle.py", line 858, in load  
>  dispatchkey  
>  File "/usr/lib/python2.7/pickle.py", line 1090, in load_global  
>  klass = self.find_class(module, name)  
> $ File "/usr/lib/python2.7/pickle.py", line 1126, in find_class  
>  klass = getattr(mod, name)  
> $ AttributeError: 'module' object has no attribute 'ClassToPickle'  
>  
>  \----------------------------------------------------------------------  
>  Ran 1 test in 0.002s  
>  
>  FAILED (errors=1)  
>  
>

And also, when I commented out a line that import ClassToPickle class (line 3
in program #3 and commented as "REQUIRED_LINE"), It doesn't work and return an
error message described below.

> E  
>  ======================================================================  
>  ERROR: testA (**main**.SomeClassTest)  
>  \----------------------------------------------------------------------  
>  Traceback (most recent call last):  
>  File "SomeClassTest.py", line 9, in testA  
>  sc = SomeClass()  
>  File "/home/tmp/pickle_problem/SomeClass.py", line 8, in **init**  
>  self.pickle = pickle.load(open("10.pickle", 'r'))  
>  File "/usr/lib/python2.7/pickle.py", line 1378, in load  
>  return Unpickler(file).load()  
>  File "/usr/lib/python2.7/pickle.py", line 858, in load  
>  dispatchkey  
>  File "/usr/lib/python2.7/pickle.py", line 1090, in load_global  
>  klass = self.find_class(module, name)  
>  File "/usr/lib/python2.7/pickle.py", line 1126, in find_class  
>  klass = getattr(mod, name)  
>  AttributeError: 'module' object has no attribute 'ClassToPickle'  
>  
>  \----------------------------------------------------------------------  
>  Ran 1 test in 0.001s  
>  
>  FAILED (errors=1)  
>

I guess the problem is about namespace in python, but I don't know what
happened exactly and what can I do for resolving it.

How can I "run as unit-test (in eclipse+pydev)" #3 program correctly,  
and run #3 program in command line without the line which imports
ClassToPickle?  
Please help me.


Answer: That because `__main__.ClassToPickle != ClassToPickle.ClassToPickle`, think of
it like this:

When you pickled the class instance of ClassToPickle in the `ClassToPickle.py`
script, the pickle module will pickle all the reference to the class which
mean it will pickle the module name where the class was defined, and because
you executed the script `ClassToPickle.py` this mean that the module will be
set to `__main__` which mean that `pickle` module will pickle
`__main__.ClassToPickle`.

And when you tried to load the pickled instance it fail because it didn't find
the instance's class which is `__main__.ClassToPickle` and not the one that
you imported using `from ClassToPickle import ClassToPickle` because this
latest is `ClassToPickle.ClassToPickle`.

A fix will be to create another script that will handle dumping instead of
doing it in `ClassToPickle.py` e.g.

    
    
    import pickle
    
    from ClassToPickle import ClassToPickle
    
    if __name__=="__main__":
        p = ClassToPickle(10)
        pickle.dump(p, open('10.pickle', 'w'))
    

