title: Feature Selection using MRMR
body: <p>I found two ways to implement MRMR for feature selection in python. The source of the paper that contains the method is: </p>
  
  <p><a href="https://www.dropbox.com/s/tr7wjpc2ik5xpxs/doc.pdf?dl=0" rel="noreferrer">https://www.dropbox.com/s/tr7wjpc2ik5xpxs/doc.pdf?dl=0</a></p>
  
  <p>This is my code for the dataset.</p>
  
  <pre><code>import numpy as np
  import pandas as pd
  from sklearn.datasets import make_classification
  
  from IPython.core.interactiveshell import InteractiveShell
  InteractiveShell.ast_node_interactivity = "all"
  
  X, y = make_classification(n_samples=10000,
                             n_features=6,
                             n_informative=3,
                             n_classes=2,
                             random_state=0,
                             shuffle=False)
  
  # Creating a dataFrame
  df = pd.DataFrame({'Feature 1':X[:,0],
                                    'Feature 2':X[:,1],
                                    'Feature 3':X[:,2],
                                    'Feature 4':X[:,3],
                                    'Feature 5':X[:,4],
                                    'Feature 6':X[:,5],
                                    'Class':y})
  
  
  y_train = df['Class']
  X_train = df.drop('Class', axis=1)
  </code></pre>
  
  <p>Method 1: Applying MRMR using pymrmr</p>
  
  <p>Contains MID and MIQ</p>
  
  <p>Which is published by the author
  The link is  <a href="https://github.com/fbrundu/pymrmr" rel="noreferrer">https://github.com/fbrundu/pymrmr</a></p>
  
  <pre><code>import pymrmr
  
  pymrmr.mRMR(df, 'MIQ',6)
  </code></pre>
  
  <blockquote>
    <p>['Feature 4', 'Feature 5', 'Feature 2', 'Feature 6', 'Feature 1',
    'Feature 3']</p>
  </blockquote>
  
  <p>or running using the second way</p>
  
  <pre><code>pymrmr.mRMR(df, 'MID',6)
  </code></pre>
  
  <blockquote>
    <p>['Feature 4', 'Feature 6', 'Feature 5', 'Feature 2', 'Feature 1',
    'Feature 3']</p>
  </blockquote>
  
  <p>Both these methods, on the above dataset yields this 2 output. Another author on GitHub claims that you can use his version to apply the MRMR method. However when I use it for the same dataset I have a different result. </p>
  
  <p>Method 2: Applying MRMR using MIFS</p>
  
  <p>Github link</p>
  
  <p><a href="https://github.com/danielhomola/mifs" rel="noreferrer">https://github.com/danielhomola/mifs</a></p>
  
  <pre><code>import mifs
  
  for i in range(1,11):
  
      feat_selector = mifs.MutualInformationFeatureSelector('MRMR',k=i)
      feat_selector.fit(X_train, y_train)
  
      # call transform() on X to filter it down to selected features
      X_filtered = feat_selector.transform(X_train.values)
  
      #Create list of features
      feature_name = X_train.columns[feat_selector.ranking_]
  
  
      print(feature_name)
  </code></pre>
  
  <p>And if you run the above iteration for all different values of i, there will come no time where both methods actually yield the same feature selection output.</p>
  
  <p>What seems to be the problem here ?</p>