context-based model and interaction terms
If you have gender data, use it. You're right that it will probably be a binary dummy feature (i.e., male = 0 or 1, and so on). You can use gender directly as a predictive feature, and/or interact it with other variables. You don't need to split your dataset by gender. Adding arbitrary information increases the noise in your set and decreases model quality. But you can learn whether gender is arbitrary or not! Add gender and compare to a model without gender to see if there's any change.