right way to use eval statement in pandas dataframe map function
Question: I have a pandas dataframe where one column is 'organization', and the content
of such column is a string with a list inside the string :

    
    
    data['organization'][0]
    Out[6] "['loony tunes']"
    
    data['organization'][1]
    Out[7] "['the three stooges']"
    

I want to substitute the string with the list which is inside the string. I
try to use map, where the function inside map is eval:

    
    
    data['organization'] = data['organization'].map(eval)
    

But the what I get is is:

    
    
    Traceback (most recent call last):
      File "C:\Users\xxx\Anaconda3\lib\site-   packages\IPython\core\interactiveshell.py", line 3035, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
      File "<ipython-input-7-3dbc0abf8c2e>", line 1, in <module>
        data['organization'] = data['organization'].map(eval)
      File "C:\Users\xxx\Anaconda3\lib\site-packages\pandas\core\series.py", line 2015, in map
        mapped = map_f(values, arg)
      File "pandas\src\inference.pyx", line 1046, in pandas.lib.map_infer     (pandas\lib.c:56983)
    TypeError: eval() arg 1 must be a string, bytes or code object
    

Thus I resorted to the following block of code, which is extremely
inefficient:

    
    
    for index, line in data['organization'].iteritems():
        print(index)
        if type(line) != str:
            data['organization'][index] = []
        try:
            data['organization'][index] = eval(data['organization'][index])
        except:
            continue
    

What am I doing wrong? how can I use eval (or a vectorized implementation)
instead of the clumsy loop above?

I thought the problem might be that some elements in pd.series
data['organization'] are not strings, so I implemented the following:

    
    
    def is_string(x):
        if type(x) != str:
            x = ''
    
    data['organization'] = data['organization'].map(is_string)
    

But I still get the same error when I try:

    
    
    data['organization'] = data['organization'].map(eval)
    

Thanks in advance.


Answer: Using eval is generally frowned upon as it **allows arbitrary python code to
be run**. So you should **strongly** prefer not to use it if possible.

In this case, you don't need to evaluate an expression, you just need to parse
the value. This means that you can use ast's
[`literal_eval`](https://docs.python.org/2/library/ast.html#ast.literal_eval):

    
    
    In [11]: s = pd.Series(["['loony tunes']", "['the three stooges']"])
    
    In [12]: from ast import literal_eval
    
    In [13]: s.apply(literal_eval)
    Out[13]:
    0          [loony tunes]
    1    [the three stooges]
    dtype: object
    
    In [14]: s.apply(literal_eval)[0]  # look, it works!
    Out[14]: ['loony tunes']
    

From the [docs](https://docs.python.org/2/library/ast.html#ast.literal_eval):

>
>     ast.literal_eval(node_or_string)
>  
>
> _Safely_ evaluate an expression node or a Unicode or Latin-1 encoded string
> containing a Python literal or container display. The string or node
> provided may only consist of the following Python literal structures:
> strings, numbers, tuples, lists, dicts, booleans, and None.

