introduction to parsing stateply predefines the initial state which all lexers start in here is the definition of the states variable for the ply parserstates (("entry""exclusive")("filename""exclusive")now that we have defined our tokens and our states we can define the regexes and functions to match the bnf t_m "\#extm udef t_info( ) "\#extinf: lexer begin("entry"return none def t_entry_seconds( ) "-?\ +, value int( value[:- ]return def t_entry_title( ) "[^\ ]+ lexer begin("filename"return def t_filename_filename( ) "[^\ ]+ lexer begin("initial"return by defaultthe tokensregexesand functions operate in the initial state howeverwe can specify that they are active in only one particular state by embedding the state' name after the t_ prefix so in this case the t_m regex and the t_info(function will match only in the initial statethe t_entry_seconds(and t_entry_title(functions will match only in the entry stateand the t_filename_filename(function will match only in the filename state the lexer' state is changed by calling the lexer object' begin(method with the new state' name as its argument so in this examplewhen we match the info token we switch to the entry statenow only the seconds and title tokens can match once we have matched title we switch to the filename stateand once we have matched filename we switch back to the initial state ready to match the next info token notice that in the case of the t_info(function we return nonethis means that the token will be discardedwhich is correct since although we must match #extinffor each entrywe don' need that text for the t_entry_seconds(functionwe strip off the trailing comma and replace the token' value with the integer number of seconds