Managed file validation on first upload only
I ended up adding a custom validator only to the new_form file upload field: 

'#type' =&gt; 'managed_file',
'#upload_validators' =&gt; array(
      'file_validate_unique' =&gt; array($file),
)


and then added a function:

function file_validate_unique($file) { 
   $errors = array();
   $filename = $file-&gt;filename;
   if (file_exists('public://' . $filename)) { 
         $errors[] = t('The file could not be uploaded because a file by that name already exists in the destination directory.');
   }
   return $errors;
}


now it only validates the new_form and outputs an error right when the user tries to upload the managed_file, before the form is even submitted.