Unsupported redo log format (0). The redo log was created before MySQL 5.7.9
You must upgrade to 5.7 as a first step.  Then you can upgrade to 8.0.

If Mac is trying to make a bigger leap, then they are doing it wrong.

You can probably see what version is installed (without hitting that problem) by doing

mysqld --version