SMOTE and oversampling with constraints
One option would be to do something more similar to bootstrapping since that would be re-sampling existing data.
Another option would be to generate extra samples then prune based on the constraints.