How can I import custom data contact references as names rather than id numbers
I see, you're right.

It would make sense that the export would have a contact reference be part of the "related contact info". Unfortunately that is not the case at this moment.

The best way for you to approach as I see it is to do the export via the CiviCRM API. It is more cumbersome to use (if you are used to using the front-end), but it has more flexibility and more possibilities.

For more info, look https://wiki.civicrm.org/confluence/display/CRMDOC/Using+the+API