title: Representation and a good similarity measure between Tweets for topic detection
body: <p>I'm planning to write a tool for Topic Detection on <em>Twitter</em>. I've been thinking about a good similarity measure (distance) between two <em>tweets</em>, and how to represent them, taking in count:</p>
  
  <ul>
  <li>The <strong><code>#hashtags</code></strong> (I think hashtags are very important when detecting topics on Twitter)</li>
  <li>The replies (if someone replies to a <em>tweet</em>, those tweets <em>could</em> be talking about the same topic, although two people could start talking about <em>samsung galaxy</em> and end talking about <em>iphone jailbreaking</em>, etc.)</li>
  </ul>
  
  <p>I'm thinking about implementing what I have so far and do some experiments. I'll implement the classic models (like <code>TF*IDF</code> and use the <em>euclidian distance</em>, <em>angle cosine</em>, etc.), and the boolean models with a few similarity measures (<em>Hamming</em>, <em>Jaccard</em>, etc.).</p>
  
  <p>Any ideas of how to adapt some existing model to <strong>Twitter</strong> or a few ideas about how to create a new one?</p>