And it takes a long time because of trivial little things going awry; random little errors falling into blind spots; naming and organizational trickiness; various forgotten tasks (also trivial, but also time-consuming); odd little snags that turn up. The developers will get through them — that’s not in question — but there’s no looming dangerous issue they can defeat proudly. Regardless, the time has melted away.
This happens even with tasks where we’ve done this before. And to be sure, there’s quite a lot of software development that falls largely into this category.
So for a more well-rounded approach to the problems of estimating: definitely, work on defining “done”, measure as well as possible, break things into small units of work, etc., but also try to minimize the psychological factors pushing in the wrong direction (more on that below).
Besides the factor I mentioned above — the embarrassment skew — there’s also what I think of as the “boring tax” — anytime you’re working on something uninteresting, it’s going to take longer simply because it’s boring. As a developer, you won’t be attempting the impossible and pursuing glory and accolades with a laser-like focus… you’ll be trudging along, and if you hop over to StackOverflow to double-check some minor technical detail, it’s pretty likely you’ll also take “just a second” to read about a much more interesting related question. You’ll wonder about other languages, frameworks, libraries, etc. that would be less boring to use for these boring tasks. You’ll think about doing some major refactoring to the entire section of the project you’re working on.
And because you’re already convinced that this is trivial (influenced by the embarrassment skew…) and you can blow through the remainder when you need to, it’s easier to give yourself permission to waste a bit more time along the way. Until the deadline is tomorrow, and the forgotten tasks pop their heads up, and you make a little mistake that causes everything to break in a really odd way.
This boring tax is easier to fight, though, and fixing that one helps to paper over the embarrassment skew as well.
DO NOT just add a percentage on to any estimate labelled “boring tax”.
That just moves out the date when the developer will realize it’s time to buckle down and get the boring stuff done.