title: sklearn clustering: calculate silhouette coefficient on TF-IDF-weigthed data
body: <p>I'd like to calculate the silhouette_score like the scikit-learn example <a href="http://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html" rel="nofollow noreferrer">silhouette_analysis</a>. </p>
  
  <pre><code>from sklearn.feature_extraction.text import TfidfVectorizer
  
  tfidf_vectorizer = TfidfVectorizer(use_idf=True)
  sampleText = []
  sampleText.append("Some text for document clustering")
  tfidf_matrix = tfidf_vectorizer.fit_transform(sampleText)
  </code></pre>
  
  <p>How do I have to convert my tfidf_matrix to do things like this:</p>
  
  <pre><code>import matplotlib.cm as cm
  from sklearn.metrics import silhouette_samples, silhouette_score
  import matplotlib.pyplot as plt
  
  
  for num_clusters in range(2,6):
      # Create a subplot with 1 row and 2 columns
      fig, (ax1, ax2) = plt.subplots(1, 2)
      fig.set_size_inches(18, 7)
  
      # The 1st subplot is the silhouette plot
      # The silhouette coefficient can range from -1, 1 but in this example all
      # lie within [-0.1, 1]
      ax1.set_xlim([-0.1, 1])
      # The (n_clusters+1)*10 is for inserting blank space between silhouette
      # plots of individual clusters, to demarcate them clearly.
      ax1.set_ylim([0, len(tfidf_matrix) + (num_clusters + 1) * 10])
  
      km = KMeans(n_clusters=num_clusters,
                  n_init=10,                        # number of iterations with different seeds
                  random_state=1                    # fixes the seed 
                 )
  
      cluster_labels = km.fit_predict(tfidf_matrix)
  
      # The silhouette_score gives the average value for all the samples.
      # This gives a perspective into the density and separation of the formed
      # clusters
      silhouette_avg = silhouette_score(tfidf_matrix, cluster_labels)
  </code></pre>