How long can a RSA public key be? If I create an RSA of 4096 bits, what would be the range of the length in bytes of the public key in DER format?

Also, will the private key always be exactly 4096 bits (512 bytes)? Does the public key length depend on the actual length of the private key, or the length you specified (4096)?
You say DER, but you tag PGP which doesn't use DER -- and whose publickey block includes a lot of things beyond the RSA values (n,e). The DER encoding of a PKCS#1 RSA publickey of 4096 bits with a conventional e of 65537 (default at least for openssl and Java) is

30 82 02 0a   02 82 02 01 (n with leading 00)   02 03 01 00 01 


with a total length of 526 octets. The DER encoding of a generic SubjectPublicKeyInfo which is always used in X.509 certificates and often in other places (including openssl and Java) adds an outer SEQUENCE and AlgorithmIdentifier and a wrapping BIT STRING giving a total length of 550 octets. If you mean something else, specify it more clearly.

For the private exponent see RSA: Most significant key bit always 1? .

But if you are concerned about the encoding of a private key, that is much more than just the exponent and (possibly) modulus; see https://stackoverflow.com/questions/5403808/private-key-length-bytes . In addition the privatekey is often encrypted, which usually expands the encrypted value and adds metadata about the encryption. 

If you have a question about a particular key encoding with a particular program or protocol/format, you need be much more specific.