Within Panels, get node ids that contain taxonomy term
Rather than getting the nid from within Panels, I just used taxonomy_select_nodes and passed the result to panels as a second argument in the URL eg panels-view/%tid/%nid.

This solved my problem.

Thanks