title: Detecting trends in a data stream in real-time
body: <p>I'm trying to detect trending topics on Twitter in real-time. What I'm doing is every time I get a tweet I assign the tweet to the cluster that talks about the same topic as the tweet. Regardless of the clustering algorithm I'm using or how I'm assigning tweets to topics, I'm unable to find how to detect a trending topic. </p>
  
  <p>My understanding or definition of the trending cluster/topic is that it's one that's getting assigned tweets more than the other clusters during a certain period of time. Or the frequency of updating the cluster size is more than the other clusters.</p>
  
  <p>How to convert that definition into actual code or a mathematical model is what I'm unable to solve. </p>
  
  <p>This is an example for how the size of a trending cluster is developing: 
  <img src="https://i.stack.imgur.com/Kez3K.png" alt="enter image description here"></p>
  
  <p>So as you see, the cluster size will be zero and then suddenly will start to increase because now it's a hot topic and tweets are being assigned to the cluster. Once the cluster is not a hot topic anymore, then the cluster size will remain relatively static.</p>