Under tmux, change window in Irssi with meta key Alt + Num?
Found the answer moments after posting here.

Apparently, Terminal.app does not escape the alt key. It can be turned on in preferences by selectin Use Option as Meta key