making multi-index in pandas dataframes in Python?
Question: I have a data set where there is a matrix of numeric values indexed by a time
variable. Each matrix is a numpy array (that can be converted into a dataframe
with columns corresponding to columns of the matrix). if i have these matrices
how can i make them into a single dataframe where each matrix has a time
index? specifically:

    
    
    # time t1
    d1 = pandas.DataFrame({"a": [1,2,3,4], "b":[10,20,30,40]})
    # time t2
    d2 = pandas.DataFrame({"a": [10,20,30,40], "b": [1,2,5,6]})
    # time t3 
    d3 = ...
    

i want to make an index called "time" to index these dataframes, and then
aggregate values from columns "a" and "b" across the time index. how can you
do this in pandas?

my attempt:

    
    
    d=pandas.DataFrame([d1,d2],index=(0, 1),columns=["time"])
    

_update_ : unutbu's solution for adding two hierarchical columns is:

` c = pd.concat([d1, d2], keys=[('t1', 'p1'), ('t2', 'p2')], names=['time',
'position']) `

my final question is how do you access this resulting structure? for example
how do you do vectorized operations across `time`, or across `position`? eg
take the average of the rows for each value of `time`.

also, how does this compare to encoding `time` and `position` into each
dataframe and using `groupby`? in other words when to use levels versus flat
columns that are grouped? here's an alternative solution using flat dataframe
with groupby:

    
    
    d1["time"] = 1
    d1["position"] = "x"
    d2["time"] = 2
    d2["position"] = "y"
    c = pandas.concat([d1, d2])
    # take mean for all time values
    c.groupby("time").apply(lambda x: np.mean(x, axis=1))
    


Answer: Given

    
    
    import pandas as pd
    
    d1 = pd.DataFrame({"a": [1,2,3,4], "b":[10,20,30,40]})
    d2 = pd.DataFrame({"a": [10,20,30,40], "b": [1,2,5,6]})
    

then [`pd.concat([d1, d2], keys=['t1',
't2'])`](http://pandas.pydata.org/pandas-
docs/stable/generated/pandas.concat.html) returns:

    
    
    In [177]: pd.concat([d1, d2], keys=['t1', 't2'])
    Out[177]: 
           a   b
    t1 0   1  10
       1   2  20
       2   3  30
       3   4  40
    t2 0  10   1
       1  20   2
       2  30   5
       3  40   6
    

* * *

If you wish to add more than one level to the new MultiIndex, you can instead
pass a _list of tuples_ to the `keys` parameter:

    
    
    In [237]: pd.concat([d1, d2], keys=[('t1', 'p1'), ('t2', 'p2')], names=['time', 'position'])
    Out[237]: 
                      a   b
    time position          
    t1   p1       0   1  10
                  1   2  20
                  2   3  30
                  3   4  40
    t2   p2       0  10   1
                  1  20   2
                  2  30   5
                  3  40   6
    

Note, it is important here that `keys` receives a list of **tuples** , rather
than a list of lists.

