Duplicated results using fields from relation File Usage: file
You need to add a File relationship for a specific field, not a File Usage relationship.

In the options for 'Add relationships' select the 'File' filter and you should see an option for each field e.g. 'File: Content using My Field'. Use this and it will give you the same data (file size etc.) but just for files attached to that field.