Looking for built-in, invertible, list-of-list-accepting constructor/deconstructor pair for pandas dataframes
Question: Are there built-in ways to construct/deconstruct a dataframe from/to a Python
list-of-Python-lists?

As far as the constructor (let's call it `make_df` for now) that I'm looking
for goes, I want to be able to write the initialization of a dataframe from
literal values, including columns of arbitrary types, in an easily-readable
form, like this:

    
    
    df = make_df([[9.75,   1],
                  [6.375,  2],
                  [9.,     3],
                  [0.25,   1],
                  [1.875,  2],
                  [3.75,   3],
                  [8.625,  1]],
                 ['d', 'i'])
    

For the deconstructor, I want to essentially recover from a dataframe `df` the
arguments one would need to pass to such `make_df` to re-create `df`.

AFAIK,

  1. [officially](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html) at least, the pandas.DataFrame constructor accepts only a numpy ndarray, a dict, or another DataFrame (and not a simple Python list-of-lists) as its first argument;
  2. the pandas.DataFrame.values property does not preserve the original data types.

I can roll my own functions to do this (e.g., see below), but I would prefer
to stick to built-in methods, if available. (The Pandas API is pretty big, and
some of its names not what I would expect, so it is quite possible that I have
missed one or both of these functions.)

* * *

FWIW, below is a hand-rolled version of what I described above, minimally
tested. (I doubt that it would be able to handle every possible corner-case.)

    
    
    import pandas as pd
    import collections as co
    import pandas.util.testing as pdt
    
    def make_df(values, columns):
        return pd.DataFrame(co.OrderedDict([(columns[i],
                                             [row[i] for row in values])
                                            for i in range(len(columns))]))
    
    def unmake_df(dataframe):
        columns = list(dataframe.columns)
        return ([[dataframe[c][i] for c in columns] for i in dataframe.index],
                columns)
    
    values = [[9.75,   1],
              [6.375,  2],
              [9.,     3],
              [0.25,   1],
              [1.875,  2],
              [3.75,   3],
              [8.625,  1]]
    columns = ['d', 'i']
    
    df = make_df(values, columns)
    

Here's what the output of the call to `make_df` above produced:

    
    
    >>> df
           d  i
    0  9.750  1
    1  6.375  2
    2  9.000  3
    3  0.250  1
    4  1.875  2
    5  3.750  3
    6  8.625  1
    

A simple check of the round-trip1:

    
    
    >>> df == make_df(*unmake_df(df))
    True
    >>> (values, columns) == unmake_df(make_df(*(values, columns)))
    True
    

BTW, this is an example of the loss of the original values' types:

    
    
    >>> df.values
    array([[ 9.75 ,  1.   ],
           [ 6.375,  2.   ],
           [ 9.   ,  3.   ],
           [ 0.25 ,  1.   ],
           [ 1.875,  2.   ],
           [ 3.75 ,  3.   ],
           [ 8.625,  1.   ]])
    

Notice how the values in the second column are no longer integers, as they
were originally.

Hence,

    
    
    >>> df == make_df(df.values, columns)
    False
    

* * *

1 In order to be able to use `==` to test for equality between dataframes
above, I resorted to a little monkey-patching:

    
    
    def pd_DataFrame___eq__(self, other):
        try:
            pdt.assert_frame_equal(self, other,
                                   check_index_type=True,
                                   check_column_type=True,
                                   check_frame_type=True)
        except:
            return False
        else:
            return True
    
    pd.DataFrame.__eq__ = pd_DataFrame___eq__
    

Without this hack, expressions of the form `dataframe_0 == dataframe_1` would
have evaluated to dataframe objects, not simple boolean values.


Answer: I'm not sure what documentation you are reading, because the link you give
explicitly says that the default constructor accepts _other list-like objects_
(one of which is a list of lists).

    
    
    In [6]: pandas.DataFrame([['a', 1], ['b', 2]])
    Out[6]: 
       0  1
    0  a  1
    1  b  2
    
    [2 rows x 2 columns]
    
    In [7]: t = pandas.DataFrame([['a', 1], ['b', 2]])
    
    In [8]: t.to_dict()
    Out[8]: {0: {0: 'a', 1: 'b'}, 1: {0: 1, 1: 2}}
    

Notice that I use `to_dict` at the end, rather than trying to get back the
original list of lists. This is because it is an ill-posed problem to get the
list arguments back (unless you make an overkill decorator or something to
actually store the ordered arguments that the constructor was called with).

The reason is that a pandas DataFrame, by default, is not an _ordered_ data
structure, at least in the column dimension. You could have permuted the order
of the column data at construction time, and you would get the "same"
DataFrame.

Since there can be many differing notions of equality between two DataFrame
(e.g. same columns even including type, or just same named columns, or some
columns and in same order, or just same columns in mixed order, etc.) --
pandas defaults to trying to be the least specific about it (Python's
principle of least astonishment).

So it would not be good design for the default or built-in constructors to
choose an overly specific idea of equality for the purposes of returning the
DataFrame back down to its arguments.

For that reason, using `to_dict` is better since the resulting keys will
encode the column information, and you can choose to check for column types or
ordering however you want to for your own application. You can even discard
the keys by iterating the `dict` and simply pumping the contents into a list
of lists if you really want to.

In other words, because order _might not_ matter among the columns, the
"inverse" of the list-of-list constructor maps backwards into a bigger set,
namely all the permutations of the same column data. So the inverse you're
looking for is not well-defined without assuming more structure -- and casual
users of a DataFrame might not want or need to make those extra assumptions to
get the invertibility.

As mentioned elsewhere, you should use `DataFrame.equals` to do equality
checking among DataFrames. The function has many options that allow you
specify the specific kind of equality testing that makes sense for your
application, while leaving the default version as a reasonably generic set of
options.

