Upgrade Postgres without doing a pg_dumpall
Since 8.4, pg_upgrade is the alternate choice to dump-restore for upgrades between major versions. It is actually linked at the end of the doc page on migration you already mentioned.

As for replaying WAL files from an older version, the doc says we can't:


  In general, log shipping between servers running different major
  PostgreSQL release levels is not possible


(in Log-Shipping Standby Servers).