Train/Validation/Testing sets for imbalanced dataset
Whit only One sample of a particular class it seems impossible to measure the classification performance on this class. So I recommend using one or more oversampling approaches to overcome the imbalance problem ([a hands-on article on it][1]). As a matter of fact, you must pay more attention to splitting the data in such a way that preserves the prior probability of each class (for example by setting the stratify argument in train_test_split). In addition, there are some considerations about the scoring method you must take into account (for example accuracy is not the best fit for scoring).
I am working in an NLP task for a classification problem. My dataset is imbalanced and some authors have 1 only text, thus I want to have this text only in the training test. As for the other authors I have to have a spliting of 70%, 15% and 15% respectivelly.
I tried to use train_test_split function from sklearn, but the results aren't good.
My dataset is a dataframe and it looks like this
Title   Preprocessed_Text   Label

Please let me know.