Why does burning wood produce methane and other hydrocarbon chains?
Under the proper conditions, the hydrocarbons present in wood are completely oxidized to $\ce{CO2}$ and water upon combustion.  In fact, this happens every day in analytical chemistry laboratories for example: a sample of wood is dried, ground and placed in a combustion chamber of oxygen with a combustion catalyst at ~$\mathrm{950^oC}$. The evolved $\ce{CO2}$ and $\ce{H2O}$ (along with other gases and ash) from the combustion can then be analyzed to quantitatively determine how much carbon and hydrogen were originally in the wood.  

Now lets consider a campfire.  Do you dry and grind your firewood, mix it with a catalyst and apply a heat source of $\mathrm{950^oC}$ under an $\ce{O2}$ atmosphere? Of course not, so why do we go through all these rigors to get a quantitative burn?  So that it all gets burned, no partial combustion allowed. And partial combustion is the key to most of the unburned hydrocarbons that you refer to. Parts of the wood are barely hot enough to burn at all, but still can thermally break down the hydrocarbons into shorter, volatile organics which are are carried away by wind or convection from the fire before the complete combustion to $\ce{CO2}$ and water can be completed.  

You also mentioned pyrolysis in your question, and the answer given in your link is certainly relevant here, and it represents one of the processes that I'm talking about. For pyrolysis, you need an oxygen-poor environment.  Going back to the campfire scenario, this largely describes the situation down in the smouldering embers on a calm night.  It's very hot, there is plenty fuel, but no process bringing in sufficient oxygen for combustion. Under these conditions, thermal decomposition rather than just partial oxidation plays a major role in producing unburned hydrocarbons.  Think about methane for example; it is certainly not partially oxidized,