Can my neural network learn conditional rules when classifying? I'm concerned that I'm attempting the impossible with my neural network. This is the scenario:
I have a 2D square world. In it, I create five circles of random size and position. I then classify one of them as the correct answer, based on the following rules:

If any circle's radius is &gt; THRESHOLD, I choose the largest circle
Otherwise, I choose the circle with the origin nearest the center

I send the inputs as serial coordinates, like this: [X0, Y0, RADIUS0, X1, Y1, RADIUS1, ...].
The output is a one-hot array, e.g. [0, 0, 1, 0, 0].
I've modeled this in TensorFlow without success. My best scoring result appears to always choose the largest circle, ignoring the else clause of the arbitrary rule.
Am I fundamentally misunderstanding the capabilities of neural networks? I've tried many (many) different configurations (layer counts, node counts, activation functions ... you name it). All of my networks have been feed-forward, so far.
Thanks in advance for any insight!

Here are some details of my network and data:

I have tried with up to 500k cases. I separate 10% for generalization checks after training, and train on the remaining 90% with a 50/50 validation split.
I've tried with the test data weighted 75% toward rule A, 50/50, and 75% toward ruleB.
I've tried 0-10 hidden layers, and neuron counts from 2 to 256 (each hidden layer gets the same number of neurons).
I change the number epochs as time allows, but generally it's 10-100. My longest runs have been several hours (with giant case numbers, and dropouts to prevent overfitting).
I've used batch sizes of 1-50.
I've tried learning rates of 0.0001 - 0.1.
I'm currently using ReLU activation, initializing bias to const(0.1) and kernel w/ heNormal. I have tried several other approaches for all three.
I standardize the inputs to center on zero w/ variance of one.
The loss function is categoricalCrossentropy.
The optimizer is Adam.
Centering the data on zero and scaling to variance of one helps with a lot of classification problems but in this case it would remove information that's needed to solve your problem as I understand it.
Another possible problem is the loss function, which you don't mention at all in your question. I would suggest something that stays fairly high when your neural network is only learning one of the two rules.