Linear classification and overlapping problem reduction
It's very difficult to tell what you're asking exactly, but one thing to keep in mind is that not al data is linearly separable. You may have to obtain more features from data or try non-linear separation techniques. SVM is one of them, but since you're not considering it, you can try polynomial separation or logistic classifiers.