Pandas variable numbers of columns to binary matrix
Question: I am currently working with a data set(csv file) which doesn't have a fixed
number of columns. however, I want to convert it to a binary matrix which have
a fixed number of columns.

as an example, current data set is like this(no headers),

    
    
    a,b,x,z,y
    b,e,w,t,u,o,s,z,i
    z,o,w
    o,p,w,z,a
    

I want this to be converted to below (first row is the header)

    
    
    a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
    
    1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1
    0,1,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,1,1,1,0,1,0,0,1
    0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,1
    1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,1
    

The main problem I am experiencing is the varied number of columns in data
set. The pseudo code or the logic I was considering is this

    
    
    header=[a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z]
    data_frame=csv file
    df=new data frame
    for each row in data_frame:
          for each item in row:
              create pandas Series 
              if item in header:
                 append '1' to Series
              else:
                 append '0' to Series
           append series to df
    

Finally the matrix should be written in to another csv file.

I have fair knowledge in python but not in pandas. Therefore, I am kindly
asking someone to give me some help with this point as I cant seems to be find
a way to do this. Thank You!


Answer: Here is one way to do it using `pd.get_dummies()`.

    
    
    import pandas as pd
    
    # read your csv data, separate must not be ',', for example, set tab `\t`
    # =======================================================================
    # I just read from clipboard
    df = pd.read_clipboard(header=None, sep='\t')
    
    df
                       0
    0          a,b,x,z,y
    1  b,e,w,t,u,o,s,z,i
    2              z,o,w
    3          o,p,w,z,a
    
    # step 1
    # =========================
    df1 = df.groupby(level=0).apply(lambda group: pd.Series(group.values.ravel().tolist()[0].split(',')))
    
    df1
    
    0  0    a
       1    b
       2    x
       3    z
       4    y
    1  0    b
       1    e
       2    w
       3    t
       4    u
           ..
       7    z
       8    i
    2  0    z
       1    o
       2    w
    3  0    o
       1    p
       2    w
       3    z
       4    a
    dtype: object
    
    
    # step 2
    # =========================
    pd.get_dummies(df1).groupby(level=0).agg(max)
    
       a  b  e ...  x  y  z
    0  1  1  0 ...  1  1  1
    1  0  1  1 ...  0  0  1
    2  0  0  0 ...  0  0  1
    3  1  0  0 ...  0  0  1
    
    [4 rows x 13 columns]
    
    # step 3, to_csv()
    # =========================
    

