Can I store media files in the private directory?
You can use the private file system with media entities. Users who have permission to view the entity they are attached to will be able to access the files. So if your file is attached to a media entity that is attached to a publicly visible node, the node is public, so the file will be open to anyone. If you add some sort of viewing permissions to the node through a contributed module or something, then the files will only be visible to those users.
It is however possible to do custom permissions, using hook_file_download().