Encrypting a binary stream with RSA + AES in counter mode
Is there anything wrong with how I'm using RSA/OAEP or AES-256-CTR? 


I wish I could answer that but I don't know enough crypto.
I hope somebody else can address this point for you.


  Is there anything I can optimize with the file format?


Nothing pops to mind. Same note as previous point.


  Any mistakes I'm not seeing?


Not really "mistakes", but I see a few minor things that could be improved.

Magic numbers

The chuck size 0xFFFF appears twice.
It would be better to put that in a variable,
perhaps CHUNK_SIZE at the package level.
That way if you ever change the value,
you only have to change in one place.

Duplicated logic

You do len(something).to_bytes(2, "big") twice.
I suppose both uses need to have consistent logic,
just like with magic numbers,
the best way to ensure consistency is to define things in one place,
rather than duplicating at two or more,
which risks inconsistent parallel edits in the future.

The approach

Once again, I'm no expert on this, but anyway...


  I'm using the SSH identity so that there's no passwords to worry about.


Makes perfect sense to me.
That is, the default private key should be protected by a passphrase,
normally managed by a key-chain manager,
and you can piggy-back on that.


  If the user's ssh private key is compromised, the data being encrypted is compromised as well.


That seems acceptable, because it's inevitable.