The caret is moved in an expected place when I add/remove a space in text I before entered in a textarea field
I wasn't able to reproduce this on a MacBook Air running Mac OS X 10.6.6 and Safari 5.0.3. When I press backspace (it's called delete on my keyboard), it doesn't cause the carat to jump to the next word.

Perhaps it's some 3rd party software interfering with your keyboard?