Pagination Not Tied to database queries
I realise the question is more than a year old...

Create a view with no content type filter.  Add fields for Title, Type, Link, and File field in that order.  In the Link field settings, check the Exclude from Display box.  In the File field settings, under No Results Behavior, use the token of the Link field in the Rewrite Output space.  The available tokens will be listed in an expandable space below the text field.  Tokens include all fields listed before the current field, which is why the order matters.  Set display to a table and turn on paging.

Views will automatically check access permissions, so this should do what you want.  This will pull in all nodes, filter by access, look for the file and use that link if it finds it, and then in the cases where that field is empty (content of another type) it will use the node link instead.