potgresql foreign key index unused for some columns on one specific server
It seems that the table publication needed a VACUUM FULL. After freeing up enough space for that and running it, new indices seem to work now.