non-linear load (rectifier) and power factor
The point you seem to be missing is that it does not require power transfer from the device back to the power line during part of the power cycle to have less than unity power factor.

There are various ways of looking at what power factor really is, although they all come out the same mathematically.  One way is the ratio of real power delivered to the product relative to the RMS voltage and current.  If the current is a sine (let's consider the voltage always a sine in this case, since the power line has such low impedance), then you have unity power factor when it is in phase with the voltage, and 0 when 90 degrees out of phase.  In the case of a sine, power does have to flow back to the line during part of the cycle to have less than unity power factor.

However, lots of other waveforms are possible.  You can have current that is always 0 or positive when the voltage is positive, or 0 or negative when the voltage is negative, but that is not a sine.  The spikes you mention caused by a full wave bridge are a good example.  Power never flows back to the power line, but yet the power factor is less than 1.  Do some examples and calculate the RMS current drawn by a full wave bridge.  You will see that the total real power drawn from the power line is less than the RMS current times the power line voltage (again, we are assuming the power line voltage is always a sine).

Another way to think of this is that losses in the tranmission system are proportional to the square of the current.  The full wave bridge draws its current in short spikes of high magnitude.  Because of the squared nature of the losses, this is worse than the same average current drawn more spread out.  I you work out that math, you realize that the way to minimize the average square of the current is to make the current be a sine in phase with the voltage.  That is the only way to achieve unity power fator.

Yet another way of looking at this, which you alluded to, is to think of the Fourier expansion of the current.  We are assuming some current waveform that repeats every power line cycle, so it has a Fourier series.  Any such repeating waveform can be expressed as a sum of a series of sine waves at the power line frequency and positive integer multiples thereof.  For example with 60 Hz power, the waveform is a sum of sines at 60 Hz, 120 Hz, 180 Hz, 240 Hz, etc.  The only question is what the amplitude and phase shift of each of these harmonics are.  It should be obvious that only the fundamental (the 60 Hz component in this example) is capable of drawing any net power from the power line, and that only to the extent it is in phase with the voltage.  Since all components are sines, each will draw power during part of the cycle and return the same power at another part of the cycle, except for the in-phase component of the fundamental.  So your way of looking at power factor as having to put power back during part of the cycle is valid if you break up the current waveform into sine wave components.  However, it is possible to have a set of sine wave components that take and return power to the power line at different times such that the net from all components at any one time is zero or positive.  The full wave bridge current is one example of such a waveform.