Python - Retrieve and replace based on a regex
Question: Basically I've imported a csv of about 17000 rows into a pandas dataframe.
There is a date column that has been imported as `int64` because data quality
is very poor. Examples of dates include `11969`, `12132001`, `1022013`, etc.
So I figure what I want to do is retrieve just the last 4 numbers from the
date column.

So the code I've used is:

    
    
    test_str = str(df['Date'])
    flags = re.MULTILINE
    p = r'\d{4}$'
    result = re.findall(p, test_str, flags)
    

When I `print(result)` only 60 of the 17000 values are returned. I'm assuming
it only assess uniques, but after a long bout of googling I can't figure it
out. Any ideas on how I can work around this?


Answer: It seems like your method actually does work (at least for the examples you
gave):

    
    
    import pandas as pd
    rng = pd.Series([11969, 12132001, 1022013, 1022013])
    test_str = str(rng)
    flags = re.MULTILINE
    p = r'\d{4}$'
    result = re.findall(p, test_str, flags)
    print(result)
    # ['1969', '2001', '2013', '2013'] # not just unique values
    

But this method of converting a `pandas` series into a string is a bizarre way
of doing it and doesn't take advantage of `pandas` inherent structure.

You might consider doing this:

    
    
    df['year_int'] = df['Date'] % 10000
    

to get the last four digits if `df['Date']` is `int64`. Or this:

    
    
    df['year_str'] = df['Date'].apply(lambda x: str(x)[-4:])
    

if you'd rather convert to a string and then take the last four characters.

    
    
    print(df)
    #        Date  year_int year_str
    # 0     11969      1969     1969
    # 1  12132001      2001     2001
    # 2   1022013      2013     2013
    # 3   1022013      2013     2013
    

