Extracting data, sorting and removing duplicates from an array using python 2.7 I have a text file data set in the form of
[['09', '07', '02', '11'], ['04', '09', '11', '04'], ['06', '07', '02', '02'], ['06', '06', '09', '09'], ['04', '11', '04', '09'], ['09', '04', '11', '04'], ['06', '09', '09', '04'] .....]

I want to extract the complete data from it, sort it and remove duplicates of the same (i.e., the same row should not appear again).
The output should look like (in rows):
11090702
11090404
07060202
09090606
09090604

I want this data for machine learning algorithm implementation.
I think with the volume of data you have, you can try something like this (using the file provided above)

import json 
with open('data1.txt', 'r') as f:
    f_json = f.read().replace('\'', '"')
    d = json.loads(f_json) 
sorted_d = list(set([''.join(sorted(x, reverse=True)) for x in d]))