title: C strtok is creating an invalid read of size 1 unable to free(token)
body: <p>Been trying to debug this simple c program:</p>
  
  <pre><code>#include &lt;stdbool.h&gt;
  #include &lt;stdlib.h&gt;
  #include &lt;stdio.h&gt;
  #include &lt;string.h&gt;
  
  #define MAX_WORD_SIZE 60
  int wordCnt = 0;
  
  int main(void){
  
  //open dictionary 
  FILE *ptr = fopen("large", "r");
  if(ptr == NULL){
    printf("unable to open %s","large");
  }
  
  //get file size 
  int fileSize;
  fseek(ptr, 0 , SEEK_END);
  fileSize=ftell(ptr) ;
  
  //get memory for file buffer (read in whole file at once, faster) 
  char * buffer = malloc(sizeof(char)*fileSize);
  
  //rewind and read in file
  fseek(ptr, 0 , SEEK_SET);
  fread(buffer, fileSize, 1, ptr);
  
  //get memory for longest word
  char * token = malloc(sizeof(char)*MAX_WORD_SIZE);
  </code></pre>
  
  <p>//here's the part causing the problem   </p>
  
  <pre><code>while (token != NULL)
  {
      if(wordCnt == 0)token = strtok(buffer, "\r\n");
      else token = strtok(NULL, "\r\n");
  
      wordCnt++;
  }
  wordCnt--;    
  
  fclose(ptr);
  free(token);
  free(buffer);
  }
  </code></pre>
  
  <p>and here are the error messages from valgrind:</p>
  
  <pre><code>valgrind ./test
  ==16233== Memcheck, a memory error detector
  ==16233== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
  ==16233== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
  ==16233== Command: ./test
  ==16233== 
  ==16233== Invalid read of size 1
  ==16233==    at 0x5E4496C: strtok (strtok.S:137)
  ==16233==    by 0x42D848: main (test.c:43)
  ==16233==  Address 0x62dd8bc is 0 bytes after a block of size 1,439,228 alloc'd
  ==16233==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
  ==16233==    by 0x42D728: main (test.c:29)
  ==16233== 
  ==16233== Invalid read of size 1
  ==16233==    at 0x5E4499C: strtok (strtok.S:163)
  ==16233==    by 0x42D848: main (test.c:43)
  ==16233==  Address 0x62dd8bc is 0 bytes after a block of size 1,439,228 alloc'd
  ==16233==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
  ==16233==    by 0x42D728: main (test.c:29)
  ==16233== 
  ==16233== 
  ==16233== HEAP SUMMARY:
  ==16233==     in use at exit: 60 bytes in 1 blocks
  ==16233==   total heap usage: 3 allocs, 2 frees, 1,439,856 bytes allocated
  ==16233== 
  ==16233== 60 bytes in 1 blocks are definitely lost in loss record 1 of 1
  ==16233==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
  ==16233==    by 0x42D7B1: main (test.c:36)
  ==16233== 
  ==16233== LEAK SUMMARY:
  ==16233==    definitely lost: 60 bytes in 1 blocks
  ==16233==    indirectly lost: 0 bytes in 0 blocks
  ==16233==      possibly lost: 0 bytes in 0 blocks
  ==16233==    still reachable: 0 bytes in 0 blocks
  ==16233==         suppressed: 0 bytes in 0 blocks
  ==16233== 
  ==16233== For counts of detected and suppressed errors, rerun with: -v
  ==16233== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
  </code></pre>