Terence Tao on Biases between consecutive primes
Biases between consecutive primes

		
 There is a very nice recent paper by Lemke Oliver and Soundararajan (complete with a popular science article about it by the consistently excellent Erica Klarreich for Quanta) about a surprising (but now satisfactorily explained) bias in the distribution of pairs of consecutive primes  when reduced to a small modulus .


This phenomenon is superficially similar to the more well known Chebyshev bias concerning the reduction of a single prime  to a small modulus , but is in fact a rather different (and much stronger) bias than the Chebyshev bias, and seems to arise from a completely different source. The Chebyshev bias asserts, roughly speaking, that a randomly selected prime  of a large magnitude  will typically (though not always) be slightly more likely to be a quadratic non-residue modulo  than a quadratic residue, but the bias is small (the difference in probabilities is only about  for typical choices of ), and certainly consistent with known or conjectured positive results such as Dirichlet’s theorem or the generalised Riemann hypothesis. The reason for the Chebyshev bias can be traced back to the von Mangoldt explicit formula which relates the distribution of the von Mangoldt function  modulo  with the zeroes of the -functions with period . This formula predicts (assuming some standard conjectures like GRH) that the von Mangoldt function  is quite unbiased modulo . The von Mangoldt function is mostly concentrated in the primes, but it also has a medium-sized contribution coming from squares of primes, which are of course all located in the quadratic residues modulo . (Cubes and higher powers of primes also make a small contribution, but these are quite negligible asymptotically.) To balance everything out, the contribution of the primes must then exhibit a small preference towards quadratic non-residues, and this is the Chebyshev bias. (See this article of Rubinstein and Sarnak for a more technical discussion of the Chebyshev bias, and this survey of Granville and Martin for an accessible introduction. The story of the Chebyshev bias is also related to Skewes’ number, once considered the largest explicit constant to naturally appear in a mathematical argument.)


The paper of Lemke Oliver and Soundararajan considers instead the distribution of the pairs  for small  and for large consecutive primes , say drawn at random from the primes comparable to some large . For sake of discussion let us just take . Then all primes  larger than  are either  or ; Chebyshev’s bias gives a very slight preference to the latter (of order , as discussed above), but apart from this, we expect the primes to be more or less equally distributed in both classes. For instance, assuming GRH, the probability that  lands in  would be , and similarly for .


In view of this, one would expect that up to errors of  or so, the pair  should be equally distributed amongst the four options , , , , thus for instance the probability that this pair is  would naively be expected to be , and similarly for the other three tuples. These assertions are not yet proven (although some non-trivial upper and lower bounds for such probabilities can be obtained from recent work of Maynard).


However, Lemke Oliver and Soundararajan argue (backed by both plausible heuristic arguments (based ultimately on the Hardy-Littlewood prime tuples conjecture), as well as substantial numerical evidence) that there is a significant bias away from the tuples  and  – informally, adjacent primes don’t like being in the same residue class! For instance, they predict that the probability of attaining  is in fact 

 with similar predictions for the other three pairs (in fact they give a somewhat more precise prediction than this). The magnitude of this bias, being comparable to , is significantly stronger than the Chebyshev bias of .

One consequence of this prediction is that the prime gaps  are slightly less likely to be divisible by  than naive random models of the primes would predict. Indeed, if the four options , , ,  all occurred with equal probability , then  should equal  with probability , and  and  with probability  each (as would be the case when taking the difference of two random numbers drawn from those integers not divisible by ); but the Lemke Oliver-Soundararajan bias predicts that the probability of  being divisible by three should be slightly lower, being approximately .


Below the fold we will give a somewhat informal justification of (a simplified version of) this phenomenon, based on the Lemke Oliver-Soundararajan calculation using the prime tuples conjecture.



To explain the Lemke Oliver-Soundararajan bias, it is convenient to relax the requirement that the primes  are consecutive, and just look at small prime differences  between primes  that are somewhat close (in the sense that  is of size , which corresponds by the prime number theorem to the mean spacing between primes), but not necessarily consecutive. (This relaxation changes some of the constants in the Lemke Oliver-Soundarajaran analysis, basically by eliminating the need to invoke the inclusion-exclusion principle, but does not affect the qualitative nature of the bias.) The naive Cramér random model for the primes (discussed for instance in this post) suggests, as a first approximation, that for any , the number of prime differences  that are equal to  with  should be on the order of . Of course, this naive model is well known to require some adjustment: most obviously, prime differences  are almost always even, so the number of solutions to  is close to zero when  is odd. A little less obviously, values of , such as , which are multiples of three should (all other things being equal) be twice as likely to be prime differences as values of  (such as ) which are not; that is to say, one expects about twice as many “sexy primes” as “twin primes“. This is ultimately because the lower prime  in a sexy prime pair  can lie in either of the two residue classes , , but the lower prime  in a twin prime pair  can only lie in the residue class  (after excluding the first twin prime pair ).


The Lemke Oliver-Soundararajan bias pushes back against this phenomenon slightly; roughly speaking, it says that a typical number  that is a multiple of  is only about  times as likely to be a prime difference as a typical number  that is a non-multiple of , for some absolute constant  (which can be computed explicitly from their work, but I will not do so here).


This bias can be established assuming the Hardy-Littlewood prime tuples conjecture (with a sufficiently good error term). This conjecture asserts, roughly speaking, that the number of solutions to  with  and some given even  is proportional to , where  is the quantity 

 The proportionality constant depends on the implicit constants in the relation , and also involves the twin prime constant 

 it will not play an important role though in our analysis, so we omit it. The reason for the  factor can be explained from the following simple calculation: if  is an odd prime, and we select two numbers  independently at random that are coprime to  (and equally likely to be in each of the  primitive residue classes mod ), then the probability that  can be calculated to be  times as large as the probability that  for any given  not divisible by . For instance, if , then  is twice as likely to equal  as it is equal , as we have already observed before.

Naively, one would expect the quantity  to be about twice as large when  is a multiple of three than when  is not a multiple of , due to the  factor of  in (1). However, it turns out that when restricting to the range , the average value of  for  a multiple of  is only about  as large as the average value of  for  not a multiple of .


If we strip out the  term in (1), creating a new function 

 then the previous bias turns out to be a consequence of an asymptotic roughly of the form 

 for some absolute constants  and all . (Actually, to avoid some artificial boundary issues one should replace the restriction  with a smoother weight such as , but we will ignore this technicality for sake of discussion.) Indeed, assuming the asymptotic (2), we have 



 whereas 

 so we see that  has a slightly lower mean (by a factor of about ) on the multiples of  than in general, which implies the corresponding claim about . We thus see that the Lemke Oliver-Soundarajan bias can be traced to the lower order term  in (2).

In the paper of Lemke Oliver and Soundararajan, the asymptotic (2) (smoothed out as discussed above) is obtained from standard complex methods, based on an analysis of the Dirichlet series 

 As it turns out, this Dirichlet series has poles at both  and  (it contains a factor of ), contributing to the  and  terms respectively. One can also establish (2) (with smoothing) using elementary number theory methods (as in this previous post); we sketch the argument as follows. We can factor  as a Dirichlet convolution 

 where  vanishes unless  is the product of distinct primes greater than or equal to , in which case 

 (with the convention ). Then we have 

 Morally speaking,  behaves like ; we can use pseudorandomness heuristics to argue that the fluctuation around this main term give a lower order contribution (and one can argue this rigorously when using smoother weights like ). The term  can be interpreted as , as per this previous post. Assuming this approximation, we obtain the approximation 

 The sequence  behaves somewhat like . As such, one expects (and can calculate)  to have an asymptotic of the form , while  has an asymptotic of the form  for some explicit constants , which gives (2).
Remark 1  One way of thinking about (2) is that the function  behaves on the average like . The bulk of the bias effect is then coming from small values  of , that is from prime gaps that are significantly smaller than average; one should not see the bias effect if one restricts to prime gaps of the typical size of . This is consistent with the general philosophy that one does not expect to see “long-range” correlations between the primes. 
	