What have I done to Preview in Yosemite?
I just tested this. Clicking the Arrow tool does indeed simply add an arrow instead of having you drag it out first.

I rarely if ever use the Annotate toolbar, so I am certain I have not made any changes that would have caused this. More likely, the upgrade to Yosemite changed how this tool works.