Terminal goes slow after install Mavericks OS
I had the same problem to and I was looking for the way to solve it.

Checking at the issue that Adam Albrecht commented, it looks like the problem was being caused by the zsh vi-mode plugin in combination with the apple provided git.

Installing git from homebrew solve the problem without having to deactivate the vi-mode plugin