title: Simple lexical analysis java program
body: <p>My little project is a lexical analysis program in which i have to take every word found in an arbitrary .java file and list every line it appears on in the file. I need to have one look up table dedicated just to the reserved words and another for all additional words found in the document. So for a program like:</p>
  
  <pre><code>    public class xxxx {
      int xyz;
      xyz = 0;
  }
  </code></pre>
  
  <p>The output should be:</p>
  
  <pre><code>Reserved words:
  class: 1
  int: 2
  public: 1
  
  Other words:
  xxxx: 1
  xyz: 2, 3
  </code></pre>
  
  <p>But there are a lot of problems with my current program and so i have no idea whats going on, so amendments to my program or a complete rewrite is welcome. I'm just trying to get a hang of the java language as a hobby so all help is welcome as long as i can understand whats going on. I'm sure there is a simple solution to this problem but my attempt didn't work :( Thanks for any help ^^</p>
  
  <pre><code>import java.io.File;
  import java.io.FileNotFoundException;
  import java.util.ArrayList;
  import java.util.HashMap;
  import java.util.Iterator;
  import java.util.List;
  import java.util.Map;
  import java.util.Scanner;
  
  public class LexicalAnalysis {
  
      private String[] keywords = { "abstract", "boolean", "byte", "case",
              "catch", "char", "class", "continue", "default", "do", "double",
              "else", "extends", "final", "finally", "float", "for", "if",
              "implements", "import", "instanceof", "int", "interface", "long",
              "native", "new", "package", "private", "protected", "public",
              "return", "short", "static", "super", "switch", "synchronized",
              "this", "throw", "throws", "transient", "try", "void", "volatile",
              "while", "false", "true", "null" };
      HashMap&lt;String, ArrayList&lt;Integer&gt;&gt; keywordsTable;
  
      HashMap&lt;String, ArrayList&lt;Integer&gt;&gt; otherWords = new HashMap&lt;String, ArrayList&lt;Integer&gt;&gt;();
  
      public LexicalAnalysis(String fileName){
  
          Scanner kb = null;
          int lineNumber = 0;
  
          try {
              kb = new Scanner(new File(fileName));
          } catch (FileNotFoundException e) {
              e.printStackTrace();
          }
  
            keywordsTable = new HashMap&lt;String, ArrayList&lt;Integer&gt;&gt;();
            for(int i = 0; i &lt; 47; i++){
                keywordsTable.put(keywords[i], new ArrayList&lt;Integer&gt;());
            }
  
          while(kb.hasNextLine()){
  
              lineNumber++;
  
              String line = kb.nextLine();
  
              String[] lineparts = line.split("\\s+|\\.+|\\;+|\\(+|\\)+|\\\"+|\\:+|\\[+|\\]+");
  
              for(String x: lineparts){
  
                  ArrayList&lt;Integer&gt; list = keywordsTable.get(x);
                  if(list == null){
                      list = otherWords.get(x);
                      if(list == null){
                          ArrayList&lt;Integer&gt; temp = new ArrayList&lt;Integer&gt;();
                          temp.add(lineNumber);
                          otherWords.put(x,temp);
                      }else{
                          otherWords.remove(x);
                          ArrayList&lt;Integer&gt; temp = new ArrayList&lt;Integer&gt;();
                          temp.add(lineNumber);
                          otherWords.put(x, temp);
                      }
                  }else{
                      keywordsTable.remove(x);
                      ArrayList&lt;Integer&gt; temp = new ArrayList&lt;Integer&gt;();
                      temp.add(lineNumber);
                      keywordsTable.put(x, temp);
                  }
              }
          }
          System.out.println("Keywords:");
          printMap(keywordsTable);
          System.out.println();
          System.out.println("Other Words:");
          printMap(otherWords);
  
      }
      public static void printMap(Map&lt;String, ArrayList&lt;Integer&gt;&gt; mp) {    
          Iterator&lt;Map.Entry&lt;String, ArrayList&lt;Integer&gt;&gt;&gt; it = mp.entrySet().iterator();    
          while (it.hasNext()) {        
              Map.Entry&lt;String, ArrayList&lt;Integer&gt;&gt; pairs = (Map.Entry&lt;String, ArrayList&lt;Integer&gt;&gt;)it.next();    
              System.out.print(pairs.getKey() + " = ");
              printList(pairs.getValue());
              System.out.println();
              it.remove();
          }
      }
      public static void printList(List x){
  
          for(Object m : x){
              System.out.print(m + ", ");
          }
  
      }
      public static void main(String args[]){
          new LexicalAnalysis("lexitest.txt");
      }
  
  
  }
  </code></pre>