By-pass door falling off track
Answering my own question, but I did eventually get the doors to behave themselves. The way I ended up doing it was this: The rollers that I got with the hardware kit and adjustable. They have one fixed screw hole and two more that are curved slots letting you not adjust the angle. I got the door in by removing the rollers from the door, placing the rollers in the track and then screwing them into the door while they were hanging from the rails. They ended up very slightly angled, but it looks fine and, more importantly, they can open and close. I had to adjust them again slightly because the door was dragging on the floor guide at the end of it's run.

It wasn't easy - an assistant would have been a great help. But I did manage to get them in and have them hanging in a way that I could move them without them falling off again. I used some shims to lift up the bottom of the door while I screwed it back in since it would be impossible to hold the door off the floor and turn the screws at the same time. Again, an assistant would have made it easier.