Single pole/single throw switch turns one light on and one light off
Someone tried to tap light B off at the switch-end of an old-school switch loop
The behavior you're describing is symptomatic of a relatively common wiring error where someone looks in a switch box, sees that there is a black and a white wire in it, doesn't pay attention to the fact that both wires go to the switch, and connects a second light across the switch at that point, assuming the white wire in the switch box is a neutral.  However, in an old-style (pre-2011 NEC) switch loop, it's not a neutral; instead, one of the wires carries power to the switch, while the other wire carries the switched-hot back up to the light, with neutral nowhere to be found in this picture.  (It terminated at light A in your situation.)
So, you'll have to either abandon the old feed for light B and feed it from somewhere that has always-hot and neutral (such as light A's box), or replace the run from the switch to light A with a /3 cable to bring neutral to the switch for use by light B.  (The latter also has the nice side effect that it brings the switch wiring up to the &quot;new style&quot; standard set by NEC 2011, where switch loops are required to bring neutral with them for use by dimmers, timers, sensors, remote-control switches, and other such &quot;smart&quot; gizmos.)
Why does it behave that way, though?
If you're wondering why this error yields the symptoms you're seeing, simply draw the resulting circuit out:

Hot goes to the junction of one side of the switch and the &quot;hot&quot; side of light B (which has a switch built in)
The other side of the switch is joined with the &quot;neutral&quot; side of light B and the &quot;hot&quot; side of light A
And the &quot;neutral&quot; side of light A is connected to neutral

As it turns out, the two lights form a series circuit, which explains the curious behaviors you see.  In particular, with the switch off, any attempt to turn light B on puts light A and light B in series, which will mean that the bulb with less resistance lights up more; with fancy LED bulbs, you get either dim glows or flickering from both bulbs, as you have observed.  Once you turn the switch on, you short out light B, which explains why light B can't turn on any longer -- the lion's share of the current is taking the express route to light A via the closed switch, instead of the congested road through light B (and its switch).