title: Using regex for tab delimited strings with RegexLineTokenizer
body: <p>I have an input file containing tab delimited strings.  On some lines there maybe multiple consecutive tabs.</p>
  
  <p>I tried using Spring's DelimitedLineTokenizer, but since there are multiple tabs, i had to consider switching to RegexLineTokenizer.</p>
  
  <p>In the RegexLineTokenizer, when i do a setRegex("\t+"), it fails to properly tokenize the lines, and fails with ArrayIndexOutOfBoundsException.</p>
  
  <p>I am not sure what am i doing wrong here.</p>
  
  <p>Here's the code:</p>
  
  <pre><code>@Bean
  public FlatFileItemReader&lt;RoutingHubInfoId&gt; routingHubIdReader() {
      System.out.println("Reading from file");
      FlatFileItemReader&lt;RoutingHubInfoId&gt; reader = new FlatFileItemReader&lt;RoutingHubInfoId&gt;();
      reader.setResource(new ClassPathResource("NAM_C4_DATA.txt"));
      DefaultLineMapper&lt;RoutingHubInfoId&gt; lineMapper = new DefaultLineMapper&lt;RoutingHubInfoId&gt;();
      RegexLineTokenizer rlt = new RegexLineTokenizer();
      rlt.setRegex("\t+");
      lineMapper.setLineTokenizer(rlt);
      lineMapper.setFieldSetMapper(new RoutingHubInfoIdMapper());
      reader.setLineMapper(lineMapper);
  
      return reader;
  }
  </code></pre>
  
  <p>The file contains the following lines (a sample)</p>
  
  <pre><code>abc def ghi
  00089BTT    IOIX    BRA
  00089BZA    BzDSA   BRA
  </code></pre>
  
  <p>Now, i tried with another piece of code, and it works.  </p>
  
  <pre><code>public class RoutingHubIdReader 
  {
      public static void main(String[] args) throws IOException, URISyntaxException
      {
          File file = new File(ClassLoader.getSystemResource("NAM_C4_DATA.txt").getFile());
          if (file.exists()) {
              System.out.println("File exists!");
          }
          else {
              System.out.println("File doesnt exist!");
          }
          LineNumberReader lnr = new LineNumberReader(new FileReader(file));
          String line;
          while ((line = lnr.readLine()) != null) {
              String[] tokens = line.split("\t+");
              if (tokens.length != 3) {
                  System.out.println(("Tokens: size: " + tokens.length + " values: " + Arrays.asList(tokens)));
              }
          }
  
      }
  }
  </code></pre>
  
  <p>It seems something simple i am missing, but not sure.  I tried also escaping the <code>\t+</code> to <code>\\t+</code>, but that didnt work either.</p>