Only Administrators can use Contact Reference Fields
You might need to provide more detail re the Permissions, but there are specific permissions for 


CiviCRM: access contact reference field
CiviCRM: access AJAX API


From memory both are required.