since the dataset doesn' have headerwe also provided the column name to the function subset of the data is shown in figure - figure - sample rows from the echonest taste profile dataset the first thing we may want to do in the dataset of this size is determine how many unique users (or songswe should consider in the original datasetwe have around million usersbut we want to determine the number of users we should consider for exampleif of all the users account for around of total play countsthen it would be good idea to focus our analysis on those users usually this can be done by summarizing the dataset by users (or by songsand getting cumulative sum of the play counts then we can find out how many users account for of the play countsetc but due to the size of the datathe cumulative summation function provided by pandas will run into trouble so we will write code to read the file line by line and extract the play count information on user (or songthis will also serve as possible method that readers can use in case the dataset size exceeds the memory available on their systems the code snippet that follows will read the file line by lineextract total play count of all the usersand persist that information for later use in [ ]output_dict {with open(data_home+'train_triplets txt'as ffor line_numberline in enumerate( )user line split('\ ')[ play_count int(line split('\ ')[ ]if user in output_dictplay_count +=output_dict[useroutput_dict update({user:play_count}output_dict update({user:play_count}output_list [{'user': ,'play_count':vfor , in output_dict items()play_count_df pd dataframe(output_listplay_count_df play_count_df sort_values(by 'play_count'ascending falseplay_count_df to_csv(path_or_buf='user_playcount_df csv'index false