How to insert a new tag with chosen module ?
I believe the Autocomplete Deluxe module aims to solve that problem.

Autocomplete Deluxe is a replacement for Chosen module. It has a similar feel to the Chosen module, but allows you to add new tags/terms.