How to set bytea_output to hex?
Sounds like PhpPgAdmin is probably doing a session-level SET bytea_output = 'escape'. 

If you open a new psql session and SHOW bytea_output; it'll be hex there.

PhpPgAdmin may offer a way to control this behaviour, or it might be hard-coded to work around compatibility issues (if it doesn't understand the hex format for bytea).