Is my protocol that uses hybrid cryptography and AES-GCM secure?
Typically things would go like this:


A generates a random AES key and encrypts it with the public key.
A encrypts the file contents using AES GCM and that key. Nonce can be random or even zero if the key is only used once. AAD can be empty unless something else needs to be authenticated with the file.
A sends over the encrypted key, the nonce if any, and the ciphertext.


The file name only matters if you need to encrypt it too (simplest: encrypt a tar archive instead) or if you need the name authenticated (include it in the AAD).