the persisted dataframe can be then loaded and used based on our requirements we can use similar strategy to extract play counts for each of the songs few lines from the dataset are shown in figure - figure - play counts for some users the first thing we want to find out about our dataset is the number of users that we will need to account for around of the play counts we have arbitrarily chosen value of to keep the dataset size manageableyou can experiment with these figures to get different sized datasets and even leverage big data processing and analysis frameworks like spark on top of hadoop to analyze the complete datasetthe following code snippet will determine the subset of users that account for this percentage of data in our case around , users account for of play countshence we will subset those users in [ ]total_play_count sum(song_count_df play_count(float(play_count_df head( = play_count sum())/total_play_count)* play_count_subset play_count_df head( = in similar waywe can determine the number of unique songs required to explain of the total play count in our casewe will find that , songs account for around of the play count this information is already great findas around of the songs are contributing to of the play count using code snippet similar to one given previouslywe can determine the subset of such songs with these songs and user subsetswe can subset our original dataset to reduce the dataset to contain only filtered users and songs the code snippet that follows uses these dataframes to filter the original dataset and then persists the resultant dataset for future uses in [ ]triplet_dataset pd read_csv(filepath_or_buffer=data_home+'train_triplets txt',sep='\ 'header=nonenames=['user','song','play_count']triplet_dataset_sub triplet_dataset[triplet_dataset user isin(user_subset