Link manually uploaded file to files-private folder
With the private file system you cannot link direct to the file path because navigation of the private files directory is prohibited.

You have to link to drupal's private file handling url, which will then check that the current user has the correct permissions to view the private file and if so grants them access and gives them  the file.

To utilise this you need to use drupal's system/files path.

So if your private files directory is files-private and you have a file files-private/filename.txt you would link to /system/files/filename.txt and drupal will handle the rest.

[Edit]
The above is still valid information, however to answer our question, yes the files need to be in the file_managed table otherwise file_file_download(), which handles private file downloads, will skip over your files because it uses file_load_multiple() to try to load your file from the database.

file_load_multiple() uses entity load and system_entity_info() defines the database table file file entities as file_managed.