title: Save memory when storing duplicate strings in a vector?
body: <p>I'm using C++ and it's STL.
  I have a large (100MB+) text file. This file just has a lot of "words" (strings separated by whitespace) like:</p>
  
  <pre><code>sdfi sidf ifids sidf assd fdfd fdfd ddd ddd
  </code></pre>
  
  <p>I need to put each of these "words" in a vector: </p>
  
  <pre><code>vector&lt;string&gt; allWordsInFile;
  </code></pre>
  
  <p>So for each word that I read from the file, i do: </p>
  
  <pre><code>allWordsInFile.push_back(word);
  </code></pre>
  
  <p>The file has a lot of duplicate words and I'm looking for ways to save memory. Each word needs to be represented in the correct position in the vector. It would be great if I could just have a list of all the words outside the vector and then just put a reference in the vector, but it's not possible to put references in a vector as far as I know. Then I thought about just storing pointers to the words, but the length of each word is so short that I don't think it will make much of a difference? (each pointer is 4 bytes on my system and most strings would probably be about the same size).</p>
  
  <p>Can someone suggest another way to approach this?</p>