Trees, and their derivative products, have been used by societies around the world for thousands of years. Contemporary construction of tall buildings from timber, in whole or in part, suggests a growing interest in the potential for building with wood at a scale not previously attainable. As wood is the only significant building material that is grown, we have a natural inclination that building in wood is good for the environment. But under what conditions is this really the case? The environmental benefits of using timber are not straightforward; although it is a natural product, a large amount of energy is used to dry and process it. Much of this can come from the biomass of the tree itself, but that requires investment in plant, which is not always possible in an industry that is widely distributed among many small producers. And what should we build with wood? Are skyscrapers in timber a good use of this natural resource, or are there other aspects of civil and structural engineering, or large-scale infrastructure, that would be a better use of wood? Here, we consider a holistic picture ranging in scale from the science of the cell wall to the engineering and global policies that could maximise forestry and timber construction as a boon to both people and the planet.