Folder icon in titlebar for iTerm2
Nevermind, found it myself. I updated to version 3.1.beta.5, and the option was under Preferences - Appearance - Window - Show proxy icon in window title bar.

You have to install the shell integrations, or manually send the required ANSI sequences. The default macOS bashrc only sends the sequences to set the icon when it detects Terminal.app.