Fecundity per woman in early humans
Infant mortality was high during the paleolithic, so the fecundity of females had to be quite a bit higher than 2 to sustain viable populations. 

A recent paper by White (2014) that explores how the ratio of young and old individuals in a population relates to demographic rates compiles data on hunter-gatherer societies from previous studies that you will probably find useful. Among other statistics, that paper reports a range in total fertility of 2.6–8.0. One of the source papers (Hewlett, 1991, pdf here) has data for individual hunter-gatherer societies (along with data from other types of pre-industrial societies). Many populations lie in the range 4-7 offspring. Note however that most (all?) of these values are from current or recent hunter-gatherer societies and not paleolithic societies. However, to give a ballpark estimate they are probably useful and fairly accurate. Also note that the child mortality rate of these societies are often between 40-50% (infant mortality ~10-30%)