How do I use Back to My Mac (to time capsule) in a multiple Airport configuration?
It turns out that only one Airport at a time is visible via Back to My Mac. I had Back to My Mac configured on both airport_downstairs and airport_upstairs. As soon as I disabled Back to My Mac on airport_upstairs and restarted both routers, airport_downstairs appeared in Back to My Mac.

I suspect that the last Airport to start up is the one that appears on Back to My Mac, though I haven't verified this.