title: Feature Importance extraction of Decision Trees (scikit-learn)
body: <p>I've been trying to get a grip on the importance of features used in a decision tree i've modelled. I'm interested in discovering the weight of each feature selected at the nodes as well as the term itself. My data is a bunch of documents.
  This is my code for the decision tree, I modified the code snippet from scikit-learn that extract (<a href="http://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importances.html" rel="nofollow">http://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importances.html</a>):</p>
  
  <pre><code>from sklearn.feature_extraction.text import TfidfVectorizer
  
  ### Feature extraction
  tfidf_vectorizer = TfidfVectorizer(stop_words=stopwords,
                                   use_idf=True, tokenizer=None, ngram_range=(1,2))#ngram_range=(1,0)
  
  tfidf_matrix = tfidf_vectorizer.fit_transform(data[:, 1]) 
  terms = tfidf_vectorizer.get_features_names()
  ### Define Decision Tree and fit
  dtclf = DecisionTreeClassifier(random_state=1234)
  
  dt = data.copy()
  
  y = dt["label"]
  X = tfidf_matrix
  
  fitdt = dtclf.fit(X, y)
  
  from sklearn.datasets import load_iris
  from sklearn import tree
  
  ### Visualize Devision Tree
  
  with open('data.dot', 'w') as file:
      tree.export_graphviz(dtclf, out_file = file, feature_names = terms)
  file.close()
  
  import subprocess
  subprocess.call(['dot', '-Tpdf', 'data.dot', '-o' 'data.pdf'])
  
  ### Extract feature importance
  
  importances = dtclf.feature_importances_
  
  indices = np.argsort(importances)[::-1]
  
  # Print the feature ranking
  print('Feature Ranking:')
  
  for f in range(tfidf_matrix.shape[1]):
      if importances[indices[f]] &gt; 0:
          print("%d. feature %d (%f)" % (f + 1, indices[f], importances[indices[f]]))
          print ("feature name: ", terms[indices[f]])
  </code></pre>
  
  <ol>
  <li>Am I correct in assuming that using terms[indices[f]] (which is the feature term vector ) will print the actual feature term used to split the tree at a certain node?</li>
  <li>The decision tree visualised with GraphViz has for instance X[30], I'm assuming this is refers to the numerical interpretation of the feature term. How do I extract the term itself so I can validate the process I deployed in #1?</li>
  </ol>
  
  <h3>Updated code</h3>
  
  <pre><code>fitdt = dtclf.fit(X, y)
  with open(...):
  tree.export_graphviz(dtclf, out_file = file, feature_names = terms)
  </code></pre>
  
  <p>Thanks in advance  </p>