Python Slicing text file into arrays based on field value
Question: I am new to Python and I want to read a text file that has three fields; `X`,
`Y` and `Time`. I want to form arrays from the x and y fields as long as the
time field is still the constant. For example:

    
    
    X    Y     Time
    1    2     100
    3    4     100
    5    6     100
    7    8     200
    9    10    200
    

therefore, for `time = 100`, I build `x_array = [1,3,5]` and `y_array =
[2,4,6]` as input to other processes and then go back to update them when time
change to `200`. Thank you for your help.


Answer: ## Solution

You can use [pandas](http://pandas.pydata.org/):

    
    
    >>> import pandas as pd
    
    >>> df = pd.read_csv('my_data.csv', delim_whitespace=True)
    
    >>> df.groupby('Time')['X'].apply(list).to_dict()
    {100: [1, 3, 5], 200: [7, 9]}
    
    >>> df.groupby('Time')['Y'].apply(list).to_dict()
    {100: [2, 4, 6], 200: [8, 10]}
    

## Explanation

This reads you file:

    
    
    df = pd.read_csv('my_data.csv', delim_whitespace=True)
    

into such a dataframe:

[![enter image description
here](http://i.stack.imgur.com/f0uVD.png)](http://i.stack.imgur.com/f0uVD.png)

Now, you group by `Time` and convert the entries in `X` into lists:

    
    
    df.groupby('Time')['X'].apply(list)
    

This gives you this pandas series:

    
    
    Time
    100    [1, 3, 5]
    200       [7, 9]
    Name: X, dtype: object
    

Finally, use `to_dict()` to convert it to a dictionary:

    
    
    >>> df.groupby('Time')['X'].apply(list).to_dict()
    {100: [1, 3, 5], 200: [7, 9]}
    

## Alternative Solution:

This gives you a different arrangement of the result:

    
    
    >>> df.groupby('Time').apply(lambda x: {'X': list(x['X']), 'Y': list(x['Y'])}).to_dict()
    {100: {'X': [1, 3, 5], 'Y': [2, 4, 6]}, 200: {'X': [7, 9], 'Y': [8, 10]}}
    

