How to control field visibility in Views based on permission?
Take global php field in view and write below logic in.

global $user;
$node=node_load($nid); 
$author=$node-&gt;uid;

if($user-&gt;uid==$author)
{
   print flag_create_link('flag_machine_name', $nid); // flag for author
}
else
{
  print flag_create_link('flag_machine_name', $nid);//flag for other users.   
}


Hope this will help.

NOTE : replace flag_machine_name with your specific flag's machine name which you want to display.