how to create tokens from CFG i have a context free grammar 

i  want to create a tokens from the language 

is there any techniques to do that ? 

for example , this CFG from Prof.Alex Aiken notes : 

 D -&gt; D ; P | D

 D -&gt; def id(ARGS) = E ; 
 ARGS -&gt; id, ARGS | id
 E -&gt; int | id | if E1 = E2 then E3 else E4 | E1 + E2 | E1 - E2 | id(E1,...,En)


how can i crate the tokens ? 

the professor explain how i do that in some inputs 

for example , if i have the input : 

x = 10 


then i have 

&lt;id.,x&gt; &lt; opp., = &gt; ......


but how i can do that from the CFG 

thank you All
The problem of creating a parse tree given a grammar and a word generated by it is known as parsing. The CYK algorithm parses all context-free grammars, but is slow. In practice, only restricted types of context-free grammars are used, and these can be parsed in linear time. The common example is LALR parsing. Perhaps you will learn more about this topic in due time.