iTerm2 - Accessibility and Full Disk Access
If you want to access special directories/files such as ~/Library/Application Support/AddressBook or ~/Pictures/Photos Library.photoslibrary, yes you need to give Full Disk Access to iTerm2. If don't, you can not access those directories.

For more information, here is the iTerm2 wiki https://gitlab.com/gnachman/iterm2/wikis/fulldiskaccess

Here is the screenshot when you don't give Full Disk Access to iTerm2.