title: ValueError: Array contains NaN or infinity in _assert_all_finite during LinearSVC
body: <p>I was trying to classify the wine data set here -<a href="http://archive.ics.uci.edu/ml/datasets/Wine+Quality" rel="nofollow noreferrer">http://archive.ics.uci.edu/ml/datasets/Wine+Quality</a>
  using logistic regression (with method ='bfgs' and l1 norm) and caught a singular value matrix error(raise LinAlgError('Singular matrix'), in-spite of full rank [which I tested using np.linalg.matrix_rank(data[train_cols].values) ] . </p>
  
  <p><strong>This is how I came to the conclusion that some features might be linear combinations of others</strong>. Towards this, I experimented of using Grid search/LinearSVC - and I get the error below, along with my code &amp; data-set .</p>
  
  <p>I can see that only 6/7 features are actually "independent" - which I interpret when comparing the rows of x_train_new[0] and x_train (so I can get which columns are redundant)</p>
  
  <pre><code>    # Train &amp; test DATA CREATION
      from sklearn.svm import LinearSVC
      import numpy, random
      import pandas as pd
      df = pd.read_csv("https://github.com/ekta1007/Predicting_wine_quality/blob/master/wine_red_dataset.csv")
  #,skiprows=0, sep=',')
  
  
      df=df.dropna(axis=1,how='any') # also tried how='all' - still get NaN errors as below
      header=list(df.columns.values) # or df.columns
      X = df[df.columns - [header[-1]]] # header[-1] = ['quality'] - this is to make the code genric enough
      Y = df[header[-1]] # df['quality']
      rows = random.sample(df.index, int(len(df)*0.7)) # indexing the rows that will be picked in the train set
      x_train, y_train = X.ix[rows],Y.ix[rows] # Fetching the data frame using indexes
      x_test,y_test  = X.drop(rows),Y.drop(rows)
  
  
  # Training the classifier using C-Support Vector Classification.
  clf = LinearSVC(C=0.01, penalty="l1", dual=False) #,tol=0.0001,fit_intercept=True, intercept_scaling=1)
  clf.fit(x_train, y_train)
  x_train_new = clf.fit_transform(x_train, y_train)
  #print x_train_new #works
  clf.predict(x_test) # does NOT work and gives NaN errors for some x_tests
  
  
  clf.score(x_test, y_test) # Does NOT work
  clf.coef_ # Works, but I am not sure, if this is OK, given huge NaN's - or does the coef's get impacted ?
  
  clf.predict(x_train)
  552   NaN
  209   NaN
  427   NaN
  288   NaN
  175   NaN
  427   NaN
  748     7
  552   NaN
  429   NaN
  [... and MORE]
  Name: quality, Length: 1119
  
  clf.predict(x_test)
  76    NaN
  287   NaN
  420     7
  812   NaN
  443     7
  420     7
  430   NaN
  373     5
  624     5
  [..and More]
  Name: quality, Length: 480
  </code></pre>
  
  <p><strong>The strange thing is that when I run clf.predict(x_train) I still see some NaN's - What am I doing wrong ?After all the model was trained using this, and this should NOT occur , right ?</strong></p>
  
  <p>According to this thread, <strong>I also checked that there are no null's in my csv file</strong> (though I relabeled the "quality' to 5 and 7 labels only (from range(3,10)
  <a href="https://stackoverflow.com/questions/18948086/how-to-fix-nan-or-infinity-issue-for-sparse-matrix-in-python">How to fix &quot;NaN or infinity&quot; issue for sparse matrix in python?</a></p>
  
  <p><strong>Also - here's the data type of x_test &amp; y_test/train...</strong></p>
  
  <pre><code>x_test
  &lt;class 'pandas.core.frame.DataFrame'&gt;
  Int64Index: 480 entries, 1 to 1596
  Data columns:
  alcohol                 480  non-null values
  chlorides               480  non-null values
  citric acid             480  non-null values
  density                 480  non-null values
  fixed acidity           480  non-null values
  free sulfur dioxide     480  non-null values
  pH                      480  non-null values
  residual sugar          480  non-null values
  sulphates               480  non-null values
  total sulfur dioxide    480  non-null values
  volatile acidity        480  non-null values
  dtypes: float64(11)
  
  y_test
  1     5
  10    5
  18    5
  21    5
  30    5
  31    7
  36    7
  40    5
  50    5
  52    7
  53    5
  55    5
  57    5
  60    5
  61    5
  [..And MORE]
  Name: quality, Length: 480
  </code></pre>
  
  <p>and Finally..</p>
  
  <pre><code>clf.score(x_test, y_test)
  
  Traceback (most recent call last):
    File "&lt;pyshell#31&gt;", line 1, in &lt;module&gt;
      clf.score(x_test, y_test)
    File "C:\Python27\lib\site-packages\sklearn\base.py", line 279, in score
      return accuracy_score(y, self.predict(X))
    File "C:\Python27\lib\site-packages\sklearn\metrics\metrics.py", line 742, in accuracy_score
      y_true, y_pred = check_arrays(y_true, y_pred)
    File "C:\Python27\Lib\site-packages\sklearn\utils\validation.py", line 215, in check_arrays
    File "C:\Python27\Lib\site-packages\sklearn\utils\validation.py", line 18, in _assert_all_finite
  ValueError: Array contains NaN or infinity.
  
  
  #I also explicitly checked for NaN's as here -:
  for i in df.columns:
      df[i].isnull()
  </code></pre>
  
  <p><strong>Tip :</strong> Please also mention if my thought process on using LinearSVC is correct, given my use case, or should I use Grid-search ?</p>
  
  <p><em>Disclaimer</em> : Parts of this code have been built on suggestions in similar contexts from StackOverflow and miscellaneous sources - My real use case is just trying to access if this method is a good fit for my scenario. That's all.</p>