Encrypting structured data (text) with RSA In "Cryptography Engineering" Bruce Schneier wrote:


  ... All of them come down to a single thing: it is very bad  to have
  any kind of structure in the numbers that RSA operates on. The 
  solution is to use a function that destroys any available structure.


Few lines lower he mentions PKCS #1 v2.1. As I understood pkcs is about padding. Does it mean that if together with random symmetric key I have some non-random data (file name) I have to xor it with some pseudo-random byte stream before encryption?

p.s. The question is crossposted from https://stackoverflow.com/questions/40194332/encrypting-strucutered-data-text-with-rsa ,  where it was voted out as off-topic since related to cryptography.
I'll assume the question is about RSA encryption of a message consisting of a random symmetric key and non-random data (a filename).

The question's "before encryption" could be referring to before using:


Textbook RSA encryption, that is changing message $m$ to $m^e\bmod N$, where $(N,e)$ is the RSA public key.
PKCS#1 v2.1 RSA encryption, that is with OAEP padding.
Hybrid encryption, which is RSA encryption as most usually practiced: RSA is used to send the secret key of a traditional symmetric cryptosystem.


What is meant by Bruce Schneier's quote is that (1) could be bad; and before applying the transformation in (1), a transformation should be applied to $m$ that makes it much like a random integer somewhat less than $N$, with little other structure.

Yes, that can be achieved with "XOR it with some pseudo-random byte stream", provided that the receiver has a way to recover than pseudo-random byte stream and undo the XOR; and that byte stream has approriate length, so that the result of the XOR is somewhat less than $N$.

In fact, this is part of what OAEP performs, and thus is built in (2). There's a caveat though: the maximum payload in (2) is reduced compared to (1). For example, with a 2048-bit (256-byte) RSA key, (1) could encipher any payload of $256-1=255$ bytes, when OAEP using SHA-256 (a 32-byte hash) limits the payload to a maximum of $256-2*32-2=190$ bytes. In both cases, the ciphertext will be 256 bytes. Thus that works for a moderate symmetric key (e.g. 16 bytes) and a small filename (up to 174 bytes). OAEP decryption will restore the payload's length.

Another option would be (3), which removes any size limit, at the price of a larger ciphertext, and often of leaking the payload's size (here, the filename's length).

Notice that (1) and (2), and many forms of (3), will not insure that what's deciphered is genuine; that's the job of authenticated encryption, which requires something secret on the sender side (beyond the message to send), contrary to RSA encryption.