recurrent neural networks val_iterator data bucketiterator(val_datadevice devicebatch_size sort_key lambda :len( textsort_within_batch false repeat falseprint(text vocab freqs most_common()[: ]output[[('the' )(' ' )('and' )('of' )('to' ('is' )('in' )(' ' )('this' )('that' )in listing - we processed couple of things that are necessary for our network for nlp use caseswe would need to tokenize and then numericalize the data as part of the text processing before using the data for the network training as you might have already guessedneural networks process only numeric data both of the aforementioned operations are neatly handled by pytorch internally we can provide an existing tokenizer--for examplespacy (an open source advanced nlp library)--and pytorch does the rest in this examplewe use custom simple one nextwe define the necessary fields (raw datafor our dataset the field class models common text-processing datatypes that can be represented by tensors alsoit holds vocab object that defines the vectors hosting the numerical representations of all words that would occur in the field our dataset has two columns"textand "label,the former being the plain english comments and the latter being numeric label ( / thuswe define text and label as two individual fields that represent our columns we add the parameter to define the tokenizing function that would be necessary on this fielda boolean flag to convert the text to lowercaseand boolean flag to indicate that the data within this field is sequential for the label fieldwe do not have sequential datahencewe set it to false