we end up with the following formatid parentid isaccepted timetoanswer score text for concrete parsing detailsplease refer to so_xml_to_tsv py and choose_ instance py it will suffice to say that in order to speed up the processwe will split the data into two files in meta jsonwe store dictionarymapping post' id to its other data (except text in the json formatso that we can read it in the proper format for examplethe score of post would reside at meta[id['score'in data tsvwe store id and textwhich we can easily read with the following methoddef fetch_posts()for line in open("data tsv"" ")post_idtext line split("\ "yield int(post_id)text strip(defining what is good answer before we can train classifier to distinguish between good and bad answerswe have to create the training data so farwe have only bunch of data what we still have to do is to define labels we couldof coursesimply use the isaccepted attribute as label after allit marks the answer that answered the question howeverthat is only the opinion of the asker naturallythe asker wants to have quick answer and accepts the first best answer if more answers are submitted over timesome of them will tend to be better than the already accepted one the askerhoweverseldom gets back to the question and changes his/her mind so we end up with many questions with accepted answers that have not been scored the highest at the other extremewe could take the best and worst scored answer per question as positive and negative examples howeverwhat do we do with questions that have only good answerssayone with two and the other with four pointsshould we really take the answer with two points as negative examplewe should settle somewhere between these extremes if we take all answers that are scored higher than zero as positive and all answers with or less points as negativewe end up with quite reasonable labels as followsall_answers [ for , in meta iteritems(if ['parentid']!=- np asarray([meta[aid]['score']> for aid in all_answers]