How to avoid errors in server log when conflict fails within a transaction before retry?
You have two unique constraints, and you say that the one on shortname causes an error only rarely.
So use
INSERT ... ON CONFLICT ON (shorturl) DO NOTHING;

Then check if you got an error and how many rows were inserted.

if you got an error, prompt the user for a new shortname

if no row was inserted, generate a new shorturl


Then you will only get error messages about duplicate shortnames in your log, which should reduce the log spam to a level that is not bothersome.