How to include validation set in the pipeline to tune parameters for an SVM?
Editing my answer -
def predict(self, train_file: str, test_file: str, lower_case: bool) -&gt; pd.DataFrame:
    &quot;Train model using sklearn pipeline&quot;
    train_df = self.read_data(train_file, lower_case)
    dev_df = self.read_data(dev_file, lower_case)
    learner = self.pipeline.fit(train_df['text'], train_df['truth'])
    # Fit the learner to the test data
    test_df = self.read_data(test_file, lower_case)
    # assuming dev_df is validset
    dev_df['pred'] = learner.predict(dev_df['text'])
    test_df['pred'] = learner.predict(test_df['text'])
    return test_df, dev_df

use two variables when you are calling the predict function to store tes and dev results