Help with contextual filters needed
The contextual filter needed to be linked to Content entity referenced from field_personstory_person_id) Content: Nid and then it worked.

This does not help me understand how to pass the argument from the panel, though ;)