Delete key doesn't work in vim on iTerm2
This is a feature of vim, although IIRC it wouldn't be enabled by default.

backspace is a setting in vim, and you can set three different parameters: indent, eol, and start

If you run :help backspace, it will tell you:

Influences the working of `&lt;BS&gt;`, `&lt;Del&gt;`, `CTRL-W` and `CTRL-U` in Insert
mode.  This is a list of items, separated by commas.  Each item allows
a way to backspace over something:

value     effect
indent    allow backspacing over autoindent
eol       allow backspacing over line breaks (join lines)
start     allow backspacing over the start of insert; CTRL-W and CTRL-U
          stop once at the start of insert.


The one that isn't set for you is start.

If you add set backspace=indent,eol,start to your .vimrc, you'll be able to backspace over anything in insert mode.