Highly imbalanced class Machine Learning
What I'd do is to keep the Unknown class and treat the Never-worked and Without-pay as Unknown. 

Imputation methods would just reinforce the bias towards the most dominant class, i.e. private in your case. Say you used a simple classifier to impute these $21$ values, it would assign them to another class based on the similarity of the other features (e.g. age, education, ...). Think on what this procedure has done though, it has replaced someone with without-pay to private or something else. This is conceptually wrong, though, especially for a model that tries to predict the income of someone.