title: How to optimize inference time for sklearn models in real time for single example
body: <p>It is a multi-class classification model with sklearn.</p>
  <p>I am using <code>OneVsOneClassifier</code> model to train and predict <code>150 intents</code>. Its a multi-class classification problem.</p>
  <p><strong>Data:</strong></p>
  <pre><code>text          intents
  
  text1         int1
  text2         int2
  </code></pre>
  <p>I convert these intents in labels using:</p>
  <pre><code>le = LabelEncoder()
  y_train = le.fit_transform(y_train)
  y_test = le.fit_transform(y_test)
  </code></pre>
  <p><strong>Expectation:</strong></p>
  <p>Without changing the training pipeline or parameters, note the inference time. Currently, it's slow, ~1second for 1 inference.
  <code>To Check if</code> : this is expected; there is a problem in setting (e.g., the kernels, the libarries); there is a bug in inference code</p>
  <p><strong>Code:</strong></p>
  <pre><code>from sklearn.feature_extraction.text import TfidfVectorizer
  from sklearn.compose import ColumnTransformer
  from sklearn.multiclass import OneVsRestClassifier, OneVsOneClassifier
  from sklearn.pipeline import Pipeline
  from sklearn.svm import SVC,LinearSVC
  
  def create_pipe(clf):
      
      # Each pipeline uses the same column transformer.  
      column_trans = ColumnTransformer(
              [('Text', TfidfVectorizer(), 'text')
               ],
              remainder='drop') 
      
      pipeline = Pipeline([('prep',column_trans),                     
                           ('clf', clf)])
       
      return pipeline
  
  def fit_and_print(pipeline):
      
      pipeline.fit(X_train, y_train)
      y_pred = pipeline.predict(X_test)
  
      print(metrics.classification_report(y_test, y_pred, 
                                          target_names=le.classes_, 
                                          digits=3))
  clf = OneVsOneClassifier(LinearSVC(random_state=42, class_weight='balanced'))
  pipeline = create_pipe(clf)
  %time fit_and_print(pipeline)
  
  # convert input to df
  
  def create_test_data(x):
      d = {'text' : x}
      df = pd.DataFrame(d, index=[0])
      return df
  
  revs=[]
  for idx in [948, 5717, 458]:
       cur = test.loc[idx, 'text']
       revs.append(cur)
  print(revs) 
  
  revs=sam['text'].values
  
  %%time
  for rev in revs:
      c_res = pipeline.predict(create_test_data(rev))
      print(rev, '=', labels[c_res[0]])
  </code></pre>
  <p>For inferencing on <code>one data at at time</code> its taking <code>~1 sec.</code> As in real, live data one data point will be given to inferencing model.</p>
  <p>But when I do <code>batch inferencing</code> it is optimised, as for <code>60 samples</code> it just takes <code>~1 second.</code></p>
  <p><strong>Is there a way to improve inference time on 1 sample? Or is this the expected behaviour?</strong></p>