Machine Learning Dataset: Easy enough for fully connected, but not easy enough for logistic regression
Here's a one-dimensional problem that should be impossible for logistic regression:


Generate x uniformly on [0, 10]
Let y = sin(2 * pi * x)


If you want a classification problem define y_disc as:


0 if y > 1/sqrt(2)
1 if 1/sqrt(2) > y > -1/sqrt(2)
2 if y &lt; -1/sqrt(2).


This is non-linear, so logistic regression should do poorly. Further, if you decide to put higher powers of x into your logistic regression classifier, I think you'll need a lot of powers to accurately represent the Taylor series near 10.

If fact, you could try adding a small noise term to y before discretizing - I suspect that will result in your logistic regressor overfitting long before it's able to accurately approximate the behavior near x=10.