I am often in a situation where I have to design and code a process/algorithm, and there are different variants on that algorithm.  For example, I am currently writing some database sychronisation code that has methods for download and upload.  Those methods are similar, but not identical.Having written the code, and got it to work, I am now refactoring it to remove the repetition.I am not so bad at writing classes which mainly hold data, in those cases I find it easier at the beginning to apply entity-relationship modelling, break the data down into classes, and I can fairly quickly see common elements that can be put into base/abstract classes.But I find it harder with algorithms/processes.My questions are: is it better to take more time to design it up-front, thus avoiding the need for refactoring? And, if so, how do you get good at doing this?I find that when I design up-front, because there is no way to test the design, I often go way off tangent and as soon as I start writing the code the design gets discarded.  In contrast, with incremental development, you can regularly test your code to ensure it works.  But the disadvantage of the way I do incremental development, is the extra time spent refactoring at the end so that I write DRY code.
Should I, and if so how do I, learn to favour upfront design with abstraction over retrospective removal of repetition?
design
For some reason, designing up front feels to our brains like it should be faster, but it rarely is.  It's much easier to factor out duplication from two working, concrete functions already in your editor than it is to factor out duplication from two ephemeral functions that haven't been written yet.The trick is to make your refactor cycle tight.  If you're doing a refactor every few minutes, that's basically indistinguishable from being really good at up front design.  On really complex algorithms, I might go an hour or two without a refactor phase.  You're doing it wrong if you ever tell your boss something like, It works, but I need a couple more days for refactoring.