How can I trace the cause of a memory spike in Postgres?
First, you should configure Linux so that it does not overcommit memory, because there's nothing you can get from a backend that is killed by the OOM killer.
Set vm.overcommit_memory = 2 and set vm.overcommit_ratio to the value calculated as
100 * (RAM - swap) / RAM

Then you should get an OOM error from PostgreSQL rather than PostgreSQL going into crash recovery.
Such an OOM error will cause a memory context dump in the PostgreSQL log file, so you can see which memory context uses all that memory. Also, you get the query that caused the error, so you can experiment with that.