GFCI (panel) breaker trips on neutral connection
A GFCI uses a current transformer (CT) to detect differences in current between the ungrounded (hot), and the grounded (neutral) conductors. Since the GFCI is tripping with only the grounded (neutral) conductor connected, that means that there's current on the grounded (neutral) conductor.  This would lead me to believe, that this neutral is tied to the neutral of another circuit somewhere. 

You'll want to make sure this isn't part of a multi-wire branch circuit. If the neutral in question enters a cable with two hot conductors, it's likely a multi-wire branch circuit. 

If not, you'll want to look for places where this circuit might be in close proximity to other circuits. A multi-gang box with two or more devices, would be a prime suspect. If you find such a location, you'll want to open it up and make sure the grounded (neutral) conductors are separated properly. It's common; especially with DIYers, to simply connect all the grounded (neutral) conductors in a box together.  However, this can lead to the exact problem you're describing.