Performing Logistic Regression with a large number of features?
Your best choice would be to use L1 regularized logistic regression (aka Lasso regression). In case you're not familiar with it, the algorithm automatically selects some of the features by penalizing those that do not lead to increased accuracy (in layman terms).
You can increase/decrease this regularization strength (it's just a parameter) till your model achieved the highest accuracy (or some other metric) on a test set or in a cross-validation procedure.
I have a dataset with 330 samples and 27 features for each sample, with a binary class problem for Logistic Regression.
According to the "rule if ten" I need at least 10 events for each feature to be included. Though, I have an imbalanced dataset, with 20% o positive class and 80% of negative class.
That gives me only 70 events, allowing approximately only 7/8 features to be included in the Logistic model.
I'd like to evaluate all the features as predictors, I don't want to hand pick any features.
So what would you suggest? Should I make all possible 7 features combinations? Should I evaluate each feature alone with an association model and then pick only the best ones for a final model?
I'm also curious about the handling of categorical and continuous features, can I mix them? If I have a categorical [0-1] and a continuous [0-100], should I normalize?