Relationship shows up on contact page but is undefined in reports, searches, and exports
I realized what the issue was.  You must be very careful in defining the relationships - A to B vs B to A.  I had different relationships defined for B to A when the A to B and B to A were the same.  For example: if you have Employer and Employee defined, you don't need to define a separate relationship for Employee to Employer -- the system handles which one is A and which one is B.