How to use an RSA public key as the primary key to a table in postgres
I would say store them as bytea, but not for that reason. Simply because pg_crypto expects the keys as bytea. That establishes a convention. They likely have good reason for it: it seems intuitively faster.

Also there is no reason to store them as a PRIMARY KEY. You can sanely just store them in an indexed field.