context-based model and interaction terms I have implemented a prediction model.
Now I am checking if I should split the model to two context-based models (men and women).
I have created a new binary feature that is relevant to only part of the women's group and it can affect the prediction accuracy for this group.
I don’t have a lot of data so I am not eager to split to context-based models.
I know that in this cases it is best to use an interaction terms features (womenxnewfeature) but this feature is only part of the women's’ group and I have heard that adding the feature as it is it’s not good.

Any suggestion on what I can do?
If you have gender data, use it. You're right that it will probably be a binary dummy feature (i.e., male = 0 or 1, and so on). You can use gender directly as a predictive feature, and/or interact it with other variables. You don't need to split your dataset by gender. Adding arbitrary information increases the noise in your set and decreases model quality. But you can learn whether gender is arbitrary or not! Add gender and compare to a model without gender to see if there's any change.