Superdeterministic hidden-variables models II: conspiracy
Indrajit Sen Antony Valentini
quant-ph physics.hist-ph


 isen@clemson.edu
Department of Physics and Astronomy,

Clemson University, Kinard Laboratory,

Clemson, SC 29634, USA
 antonyv@clemson.edu
Department of Physics and Astronomy,

Clemson University, Kinard Laboratory,

Clemson, SC 29634, USA
Augustus College,

14 Augustus Road,

London SW19 6LN UK



We prove that superdeterministic models of quantum mechanics are conspiratorial in a mathematically well-defined sense, by further development of the ideas presented in a previous article 𝒜. We consider a Bell scenario where, in each run and at each wing, the experimenter chooses one of N devices to determine the local measurement setting. We prove, without assuming any features of quantum statistics, that superdeterministic models of this scenario must have a finely-tuned distribution of hidden variables. Specifically, fine-tuning is required so that the measurement statistics depend on the measurement settings but not on the details of how the settings are chosen. We quantify this as the overhead fine-tuning F of the model, and show that F > 0 (corresponding to `fine-tuned') for any N >1. The notion of fine-tuning assumes that arbitrary (`nonequilibrium') hidden-variables distributions are possible in principle. We also show how to quantify superdeterministic conspiracy without using nonequilibrium. This second approach is based on the fact that superdeterministic correlations can mimic actual signalling. We argue that an analogous situation occurs in equilibrium where, for every run, the devices that the hidden variables are correlated with are coincidentally the same as the devices in fact used. This results in extremely large superdeterministic correlations, which we quantify as a drop of an appropriately defined formal entropy. Nonlocal and retrocausal models turn out to be non-conspiratorial according to both approaches.

Superdeterministic hidden-variables models II: conspiracy
    Antony Valentini
    December 30, 2023
=========================================================





§ INTRODUCTION

In a previous article 𝒜 <cit.>, we gave a broad overview of superdeterministic models and criticisms thereof. We also discussed the properties of these models in nonequilibrium. In particular, we showed that nonequilibrium extensions of superdeterministic models have two striking properties. First, the measurement statistics have a general dependence on the mechanism used to determine the measurement settings. This implies that the equilibrium distribution has to be finely-tuned such that these effects disappear in practice. Second, although these models violate marginal-independence (that is, they violate formal no-signalling), there is no actual signalling. Instead, the local outcomes and distant settings are only statistically correlated due to initial conditions. We argued that this mimicking of a signal using statistical correlations is intuitively conspiratorial. In this paper, we further develop these ideas mathematically to give two separate ways to quantify the conspiratorial character of superdeterministic models.
 

It is useful to discuss here the relation between experimental observations, fine tuning, and the nonequilibrium framework. Although all possible distributions are allowed in nonequilibrium, experimental observations are important in the framework. A useful example here would be the astrophysical and cosmological tests of nonequilibrium distributions that have been proposed in pilot-wave theory <cit.>. The framework, however, considers it unsatisfactory to fine tune the initial hidden-variables distribution to reproduce the experimental observations. A classic example of this is signal locality (marginal independence or formal no-signalling), which has been experimentally confirmed in numerous experiments. The usual explanation of signal locality involves fine tuning of the initial distribution <cit.>. The same conclusion was arrived at later by other workers applying causal discovery algorithms to Bell correlations <cit.>. In this paper, we use the nonequilibrium framework to ask whether superdeterministic models need to be fine tuned so that the measurement statistics depend on the measurement settings but not on the details of how the settings are chosen. 


The paper is structured as follows. In section <ref> we define our experimental setup and show that superdeterministic models of our scenario must be fine tuned so that the measurement statistics do not depend on the details of how the measurement settings are chosen. We also show that nonlocal and retrocausal models of our scenario do not have to be similarly fine tuned. In section <ref>, we quantify the fine tuning required for different kinds of superdeterministic models by introducing a fine tuning parameter F. In section <ref>, we propose a different approach to quantifying the conspiracy which foregoes any use of nonequilibrium distributions. We discuss our results in section <ref>. 



§ SUPERDETERMINISTIC CONSPIRACY AS FINE TUNING
 
Consider the standard Bell scenario <cit.>, where two spin-1/2 particles prepared in the spin-singlet state are each subjected to one of two local spin measurements, say M_x (corresponding to σ̂_x̂) or M_z (corresponding to σ̂_ẑ), in a spacelike separated manner. The local measurement settings at both wings are chosen using a setting mechanism, which is defined as any physical system that outputs a measurement setting. A simple example is a die whose faces have been painted either M_x or M_z. An experimenter can roll the die for each run and use its output to choose the local measurement setting. Other examples of setting mechanisms can include coins, random number generators, humans choosing measurement settings, and so on. 


We suppose that the experimenter at each wing has a setup consisting of N different setting mechanisms, which collectively give N outputs for each run. The experimenter then chooses one of the outputs and uses it as the actual measurement setting for that run (how he or she makes this choice has no relevance to our argument, so it can be specified arbitrarily). Nevertheless, the outputs of all the N setting mechanisms are recorded for each run at both the wings.
 

We label the hidden variables that determine the measurement outcomes at both wings by λ. The output of the i^th(j^th) setting mechanism at wing A (B) is labelled by α_i (β_j), where i, j ∈{1,2...N} and α_i, β_j ∈{M_x, M_z}. We label the experimenters' choice of the outputs in the following manner: if the experimenter at wing A (B) chooses the i^th (j^th) setting mechanism, then we have γ_A = i (γ_B = j). Lastly, we label the actual measurement settings at wing A (B) by M_A (M_B), where M_A, M_B ∈{M_x, M_z}.


We know that, in a superdeterministic model, λ is correlated with the measurement settings. As these models are deterministic, it follows that λ is correlated with the physical variables that determine the measurement settings. In our scenario, the measurement settings are functions of the N setting mechanism outputs and of the experimenter's choice for each run (at both wings). Therefore, in general, λ will be correlated with {α} ({β}) and γ_A (γ_B) at wing A (B), where {α} ={α_1, α_2,...α_N} and {β}={β_1, β_2,... β_N}. A superdeterministic model of our scenario is schematically illustrated in Fig. 1. Note that, as in 𝒜, we have not used the interventionist notion of causation <cit.> to define the causal relationships in the figure, as the model is deterministic. For a deterministic model, the relation between the effect (defined to be in the future) and the cause (defined to be in the past) is that of functional dependence. This definition uses only the concepts of temporal ordering and functional dependence.




The measurement statistics will be encoded in the distribution p(O_A, O_B| M_A, M_B). For both type I and type II models (see 𝒜 and Fig. 1 below), this distribution can be expanded as 

    p(O_A, O_B| M_A, M_B) = ∑_λ∑_{α}, γ_A∑_{β}, γ_B p(O_A, O_B, λ, {α}, γ_A, {β}, γ_B | M_A, M_B)
       = ∑_λ∑_{α}, γ_A∑_{β}, γ_Bp(O_A, O_B| λ, M_A, M_B) p(λ | M_A, M_B, {α}, γ_A, {β}, γ_B) p({α}, γ_A, {β}, γ_B|M_A, M_B)

where we have used p(O_A, O_B| λ,  {α}, γ_A, {β}, γ_B, M_A, M_B) = p(O_A, O_B| λ, M_A, M_B) as the measurement outcomes depend only the measurement settings given the hidden variables. We note that the measurement statistics depend, in general, on {α}, γ_A and {β}, γ_B as the hidden-variables distribution p(λ| M_A, M_B, {α}, γ_A, {β}, γ_B) has a dependence on these variables. For example, suppose the variables are ({α}', γ_A', {β}', γ_B') for a particular set of runs and ({α}”, γ_A”, {β}”, γ_B”) for another set. Let the measurement settings for both the sets be the same. That is, M_A({α}', γ_A') = M_A({α}”, γ_A”) and M_B({β}', γ_B') = M_B({β}”, γ_B”). In general, the measurement statistics for both the sets will differ. This can be for two different reasons. First, λ may be correlated with different setting mechanisms differently. We know that, when γ_A = i (γ_A = j), λ is correlated with M_A due to its correlation with α_i (α_j). In general, λ may be correlated with α_i and α_j differently. Therefore, if γ_A' = i and γ_A” =j, (j ≠ i), then the hidden-variables distribution will be different for both the sets, leading to different measurement statistics in general. Second, λ may be non-trivially correlated with the unused setting mechanisms. For example, suppose γ_A' =γ_A” = i, but α_j' ≠α_j”. If λ is non-trivially correlated with α_j, then the distribution will be different for both the sets, leading to different measurement statistics again in general. Many such sets can be similarly constructed that have the same measurement settings M_A and M_B but different {α}, γ_A, {β}, γ_B values. The measurement statistics for these sets will be the same if and only if

    p(O_A, O_B| M_A, M_B, {α}', γ_A', {β}', γ_B') = p(O_A, O_B| M_A, M_B, {α}”, γ_A”, {β}”, γ_B”)

where M_A({α}', γ_A') = M_A({α}”, γ_A”) and M_B({β}', γ_B') = M_B({β}”, γ_B”). Equation (<ref>) formalises the condition that the measurement statistics depend only on the measurement settings. Expanding both distributions over λ, we have

    ∑_λ p(O_A, O_B| λ, M_A, M_B) p(λ| M_A, M_B, {α}', γ_A', {β}', γ_B') = ∑_λ p(O_A, O_B| λ, M_A, M_B) p(λ| M_A, M_B, {α}”, γ_A”, {β}”, γ_B”)

It is clear that condition (<ref>) will not be satisfied in general for an arbitrary hidden-variables distribution p(λ| M_A, M_B, {α}, γ_A, {β}, γ_B). Therefore, we conclude that superdeterministic models of our scenario must be fine tuned. Note that this fine tuning is not needed to reproduce any particular feature of the quantum predictions, such as marginal independence (formal no signalling), non contextuality, and so on. It is only needed to satisfy equation (<ref>). 


On the other hand, one can show that retrocausal and nonlocal models of our scenario do not need this finetuning[All hidden-variables models need to be fine-tuned in order to reproduce marginal independence (formal no signalling) <cit.>, unless a dynamical explanation of finetuning is given <cit.>. The fine tuning we are discussing here refers only to that required to satisfy condition (<ref>).]. Let us first consider retrocausal models of our scenario. The causal order diagram for such a model is shown in Fig. 2 a). Note that we have used the interventionist notion of causation <cit.> to define causal relationships in Fig. 2 a) as retrocausal models are not deterministic (given the past conditions alone). The measurement setting causally affects λ backwards in time in these models. The measurement setting, in turn, is causally determined by the setting mechanisms and the the experimenters' choices forwards in time. In this way, λ is correlated with the variables {α}, γ_A at wing A and {β}, γ_B at wing B. Intuitively, we expect that only the measurement setting, not the exact manner in which the measurement setting is chosen, will be important. This is because the setting of the measuring apparatus, regardless of how it is selected, causally affects λ (backwards in time). Therefore, the correlation between λ and the measurement settings does not depend on how the settings are chosen, that is, p(λ|M_B, {α}, γ_A, {β}, γ_B) = p(λ|M_B) for the retrocausal model in Fig. 2 a). We can understand this more explicitly by considering the joint distribution of the model parameters

    p(O_A, O_B, λ, M_A, M_B, {α}, γ_A, {β}, γ_B) =    p(O_A|λ, M_A) p(O_B|λ, M_B) p(λ| M_B) p(M_A|{α}, γ_A) ×
       p(M_B| {β}, γ_B) p({α}) p(γ_A) p({β}) p(γ_B)

We can sum over the variables {α}, γ_A at wing A and {β}, γ_B at wing B to yield

    ∑_{α}, γ_A∑_{β}, γ_B p(O_A, O_B, λ, M_A, M_B, {α}, γ_A, {β}, γ_B) = p(O_A|λ, M_A) p(O_B|λ, M_B) p(λ|M_B) p(M_A) p(M_B)

This means that we can always build a reduced model where only the parameters p(O_A|λ, M_A), p(O_B|λ, M_B), p(λ|M_B) and p(M_A), p(M_B) are defined. The variables {α}, γ_A, {β}, γ_B do not need to be explicitly included in the model. Therefore, the model does not have to be fine tuned with respect to these variables and condition (<ref>) is automatically satisfied. 


Next consider a nonlocal model of our scenario. The causal order diagram will be as shown in Fig. 2 b). The joint distribution of the model parameters is 

    p(O_A, O_B, λ, M_A, M_B, {α}, γ_A, {β}, γ_B) =    p(O_A|λ, M_A, M_B) p(O_B|λ, M_B) p(λ) p(M_A|{α}, γ_A) ×
       p(M_B| {β}, γ_B) p({α}) p(γ_A) p({β}) p(γ_B)

Similarly to the retrocausal model, we can sum over the variables {α}, γ_A at wing A and {β}, γ_B at wing B to yield

    ∑_{α}, γ_A∑_{β}, γ_B p(O_A, O_B, λ, M_A, M_B, {α}, γ_A, {β}, γ_B) = p(O_A|λ, M_A, M_B) p(O_B|λ, M_B) p(λ) p(M_A) p(M_B)

Therefore, the only necessary model parameters are p(O_A|λ, M_A, M_B), p(O_B|λ, M_B), p(λ) and p(M_A), p(M_B). A nonlocal model of our scenario, therefore, does not have to be fine tuned with respect to the variables {α}, γ_A, {β}, γ_B in order to satisfy condition (<ref>). Unlike for retrocausal and nonlocal models, the variables {α}, {β}, γ_A, and γ_B cannot be summed over similarly for superdeterministic models of our scenario. The causal order diagram for a superdeterministic model of our scenario (see Fig. 1) implies that p(λ|M_A, M_B, {α}, γ_A, {β}, γ_B) = p(λ|{α}, γ_A, {β}, γ_B). 


The measurement statistics for superdeterministic models depend crucially on the correlation between λ and the variables {α}, {β}, γ_A, and γ_B (given M_A and M_B). Therefore, fine tuning is required so that the measurement statistics depend on the measurement settings but not on the details of how the settings are chosen. Note that we have not imposed any features of quantum statistics, that is, any features of the Born rule, to argue that superdeterministic models are fine tuned. In principle, we can impose the additional requirement that the superdeterministic model reproduces the Born rule. This will require an additional fine tuning. However, this is not relevant to the question we are asking in this paper: whether superdeterministic models require fine tuning so that the measurement statistics depend on the measurement settings but not on the details of how the settings are chosen. Our results show that superdeterministic models require fine tuning regardless of whether or not they reproduce quantum statistics, in sharp contrast with retrocausal or nonlocal models. Note further that we have not imposed the condition of locality either.






§ QUANTIFICATION OF FINE TUNING
 
In this section, we quantify the degree of fine tuning required for superdeterministic models of our scenario to satisfy condition (<ref>). We first assume that 

    p(λ| M_A, M_B, {α}, {β}, γ_A, γ_B) = l/L ∀λ

for some l ∈{0,1,2...L}, where L is a very large positive integer. Although we are assuming that λ is a discrete variable, our argument can be applied to discrete approximations of any continuous distribution, where the approximation can be made arbitrarily close by increasing L. Let the number of possible values of λ be Λ.


We can represent the hidden-variable distribution by using the probability simplex (p ∈ℛ^Λ_+ | ∑_λ p(λ|M_A, M_B, {α}, {β}, γ_A, γ_B) =1, p(λ|M_A, M_B, {α}, {β}, γ_A, γ_B) ≥ 0  ). In the simplex notation, a particular configuration of the distribution is represented by a single point p on a (Λ -1) dimensional object. By a configuration, we mean a particular value of the distribution for all λ's. For example, for Λ = 3, the different configurations of the distribution can be represented by points on a plane (see Fig. 3). Each point on the plane specifies a particular set of values {p(λ_1|M_A, M_B, {α}, {β}, γ_A, γ_B), p(λ_2|M_A, M_B, {α}, {β}, γ_A, γ_B), p(λ_3|M_A, M_B, {α}, {β}, γ_A, γ_B)} for the distribution p(λ|M_A, M_B, {α}, {β}, γ_A, γ_B). Let the number of points on the simplex that satisfy (<ref>) for all λ be labelled by V(Λ, L). In nonequilibrium, there are V(Λ, L) possible configurations of each distribution. Note that V(Λ, L) scales with Λ as V(Λ, L) = (L + Λ -1)!/ L! (Λ-1)!.


Let Ω be the total number of independent distributions in the model, including nonequilibrium distributions. For example, consider Ω for our scenario. We know that there are N possible γ_A and γ_B values. Each α or β can take one of 2 values (M_x or M_z). The number of possible {α} or {β} values, therefore, is 2^N. In total, there will be N^2 2^2N independent distributions p(λ| M_A, M_B, {α}, {β}, γ_A, γ_B). Thus, Ω = N^2 2^2N for our scenario. Each distribution can take one of V(Λ, L) configurations (points on the simplex). The Ω distributions can have V(Λ, L)^Ω different configurations in total. Let the model impose certain constraints on the configurations to satisfy condition (<ref>). Suppose, as a result of these constraints, the total number of configurations is reduced to N_f. Then we define the quantity

    F =  1 - N_f/V(Λ, L)^Ω

as the overhead fine-tuning of the hidden-variables model. If F=0 we call the model completely general, as the total number of configurations after applying the constraints is the same as before. On the other hand, if F=1 we call the model completely fine-tuned, as there are no configurations left after imposing the constraints. Any value of F outside [0,1] reflects inconsistency in the model. We now use this definition to quantify the fine tuning of superdeterministic models of our scenario.




We first consider the kind of superdeterministic model usually proposed in the literature, where there is single distribution p(λ| M_A, M_B). One can obtain such a model by imposing the following constraints on the general distribution p(λ| M_A, M_B, {α}, {β}, γ_A, γ_B):

    p(λ| M_A, M_B, {α}, {β}, γ_A=i, γ_B=j)    = p(λ| M_A, M_B, α_i, β_j, γ_A=i, γ_B=j) 
    
    p(λ| M_A, M_B, α_i, β_j, γ_A=i, γ_B=j)    = p(λ| M_A, M_B)

for all i, j. Note that M_A = M_A({α}, γ_A) and M_B = M_B({β}, γ_B). Constraint (<ref>) implies that λ is correlated with the output of only the setting mechanism actually used for all runs. Constraint (<ref>) implies that λ is correlated with the output of each setting mechanism in the same way. Note that p(λ| M_A, M_B) need not be the equilibrium distribution, as we have not imposed the condition that the model reproduces the predictions of quantum mechanics. It is clear that a superdeterministic model that satisfies equations (<ref>) and (<ref>) will satisfy equation (<ref>), and thereby condition (<ref>). Let us find out the value of F for such a model.


The model (after being constrained) has only four independent distributions p(λ| M_A, M_B), as we assume that M_A and M_B can each take one of only two values: M_x or M_z. Each of the four distributions can occupy any of the V(Λ, L) points on the simplex, as (<ref>) is satisfied automatically due to (<ref>) and (<ref>). Therefore, N_f = V(Λ, L)^4. The fine-tuning parameter F will be

    F    = 1 - V(Λ, L)^4/V(Λ, L)^Ω
       = 1 - V(Λ, L)^4 -N^2 2^2N,

as we know that Ω = N^2 2^2N. From the `excess-baggage' theorem <cit.>, we know that Λ→∞. For N= 1, F= 0 (there is no fine tuning). However, for any N > 1, F = 1 in the limit V(Λ, L) →∞ (we know that as Λ→∞, V(Λ, L) →∞). Therefore, the model is arbitrarily close to complete fine tuning for any N > 1. This means that imposing the constraints (<ref>) and (<ref>) to reproduce the features a) and b) is a bad strategy from a fine tuning perspective. 


Let us now consider more general superdeterministic models. We know that equation (<ref>) is the minimum constraint that needs to be satisfied in order to satisfy (<ref>). Let us, therefore, determine F for a superdeterministic model of our scenario with (<ref>) as the only constraint. This will be the minimum fine tuning required for a general superdeterministic model of our scenario. To begin, we note that the total number of independent distributions does not change as a result of (<ref>). Out of these Ω distributions, there are Ω/4 distributions corresponding to a particular value of (M_A, M_B). That is, there are Ω/4 distributions corresponding to each of (M_x, M_x), (M_x, M_z), (M_z, M_x) and (M_z, M_z). We need to ensure that equation (<ref>) is satisfied for each value of (M_A, M_B). Consider the Ω/4 distributions that correspond to a particular value of (M_A, M_B). Consider a single distribution out of these possibilities. Let this distribution be unconstrained, so that the number of possible configurations of this distribution is V(Λ, L). Let us label the j^th configuration of the unconstrained distribution by p^j(λ| M_A, M_B, {α}^u, {β}^u, γ_A^u, γ_B^u), where j ∈{1, 2, 3....V(Λ, L)}. Consider another (constrained) distribution p(λ| M_A, M_B, {α}^c, {β}^c, γ_A^c, γ_B^c) out of the remaining Ω/4 - 1 possible distributions that correspond to (M_A, M_B). Equation (<ref>) then implies

    ∑_λ p(O_A, O_B| λ, M_A, M_B) p^j(λ| M_A, M_B, {α}^u, γ_A^u, {β}^u, γ_B^u) = ∑_λ p(O_A, O_B| λ, M_A, M_B) p(λ| M_A, M_B, {α}^c, γ_A^c, {β}^c, γ_B^c)

Clearly, the remaining Ω/4 - 1 distributions will have to be constrained in order to satisfy (<ref>). Let the number of configurations of each constrained distribution (that satisfies (<ref>) for a given j^th configuration of the unconstrained distribution) be v^j_AB(Λ, L), where 0 < v^j_AB(Λ, L)/V(Λ, L) < 1. The total number of configurations of the distributions corresponding to (M_A, M_B) will then be ∑_j=1^V(Λ, L) v^j_AB(Λ, L)^Ω/4 -1.  The fine tuning parameter F will be

    F = 1 -  ∏_A, B∑_j=1^V(Λ, L) v^j_AB(Λ, L)^Ω/4 -1/V(Λ, L)^Ω

We know that v^j_AB(Λ, L)/V(Λ, L) < 1 ∀ j, A, B. This implies that 

    (v^j_AB(Λ, L)/V(Λ, L) )^Ω/4 -1 < 1
    ⇒   ∑_j=1^V(Λ, L) (v^j_AB(Λ, L)/V(Λ, L) )^Ω/4 -1 < V(Λ, L)

This suggests that we write the fine-tuning parameter in the following form

    F    = 1 - ∏_A, B∑_j=1^V(Λ, L) (v^j_AB(Λ, L)/V(Λ, L) )^Ω/4 -1/V(Λ, L)
       = 1  - ∏_A, B∑_j=1^V(Λ, L) (v^j_AB(Λ, L)/V(Λ, L) )^N^22^2N-2 -1/V(Λ, L)

For N =1, F =0 (there is no fine tuning). From equations (<ref>) and (<ref>), we see that 0< F <1 for any N >1. This implies that for finite N, unlike the superdeterministic model that satisfied (<ref>) and (<ref>), the model is not completely fine tuned (but still fine tuned). Lastly, F = 1 in the limit N →∞. This is because (v^j_AB(Λ, L)/V(Λ, L) )^m → 0 as m →∞. Note that both v^j_AB(Λ, L) and V(Λ, L) do not depend on N, and that the result F =1 does not depend on the condition Λ→∞.




§ CONSPIRACY AS SPONTANEOUS ENTROPY DROP
 
In the previous section, we quantified the conspiratorial character of superdeterministic models of our scenario in terms of a fine tuning paramter F. The definition of F rests on the concept of quantum nonequilibrium since V(Λ) includes contributions from all possible configurations of the hidden-variable distribution. For example, consider two distributions p(λ|{α}', {β}', γ_A', γ_B') and p(λ|{α}”, {β}”, γ_A”, γ_B”), where M_A({α}', γ_A') = M_A({α}”, γ_A”) and M_B({β}', γ_B') = M_B({β}”, γ_B”). For most of the configurations of p(λ|{α}', {β}', γ_A', γ_B') and p(λ|{α}”, {β}”, γ_A”, γ_B”), equation (<ref>) will not be satisfied. But the definition of F includes all these configurations as well in the quantity V(Λ)^Ω. This is natural in a nonequilibrium framework, where all hidden-variable distributions are possible in principle. In this section, we give a quantification of superdeterministic conspiracy without using nonequilibrium distributions[Although there are compelling theoretical reasons <cit.> that motivate nonequilibrium extensions of hidden-variable models, a debate about nonequilibrium, for the purposes of this article, is secondary to proving that superdeterministic models are unequivocally conspiratorial.]. This approach is more general also in the sense that λ and p(λ|{α}, {β}, γ_A, γ_B) do not have to be discrete. On the other hand, this approach is more limited in that it is applicable only to superdeterministic models that satisfy the constraint (<ref>). This approach is, however, still widely applicable as all the superdeterministic models that have been proposed in the literature so far posit a single distribution p(λ| M_A, M_B). Therefore, those models satisfy both constraints (<ref>) and (<ref>) when extended to our scenario.


To begin, we first remind ourselves of the conspiratorial nature of apparent-signalling in superdeterminism, as discussed in 𝒜. The violation of marginal-independence (or formal no-signalling) does not imply actual signalling in a superdeterministic context. The local marginals and the distant settings are correlated, but the distant setting does not causally affect the local marginal. The correlation is set up by the initial conditions. This appears to be a signal, because the distant setting can be controlled by an experimenter and the local marginal is correlated with the experimenter's decisions. We, therefore, term superdeterministic signalling as apparent signalling. We will now argue that an analogous situation occurs in superdeterministic models even in equilibrium. 


Consider a superdeterministic model of our scenario that satisfies (<ref>). Equation (<ref>) implies that there are runs for which λ is correlated only with the outputs α_i and β_j (for a given pair (i, j)), and further, that these runs occur only when the experimenters choose γ_A =i and γ_B = j. We may say that these runs belong to a sub-ensemble E = (i, j) having the hidden-variables distribution ρ(λ| α_i, β_j, γ_A = i, γ_B = j). There will be N^2 such sub-ensembles. However, although a run belonging to (i, j) only occurs when the experimenters choose γ_A =i and γ_B = j, there is no causal relationship between γ_A, γ_B and the sub-ensemble E. Therefore, as in the case of apparent signalling, although it appears that the experimenters' choices determine which sub-ensemble a particular run belongs to, it is simply the case that the initial conditions have arranged a functional relationship E = (γ_A, γ_B).


Note that this is true regardless of whether γ_A, γ_B are determined by human experimenters, computer algorithms or any other device. It is also independent of N. Therefore, we have a situation where each setting mechanism is coincidentally used only when it is correlated with the hidden variables. This appears conspiratorial intuitively. Below, we quantify this intuition as a spontaneous decrease of an appropriately-defined subjective `entropy' at the hidden-variables level.


Consider a superdeterministic model of our scenario that satisfies (<ref>). Let there be N_0 runs in total. Each run belongs to a particular sub-ensemble E =(i, j) out of N^2 possibilities. The number of possible sequences of sub-ensembles over the N_0 runs is 

    W = N^2N_0

In quantum equilibrium, the experimenters cannot know which sequence, out of the W possibilities, is actually the case. Suppose that by some means the experimenters find out the exact sequence. Then, they can choose γ_A and γ_B values that do not match this sequence. In that case, the Bell correlations will not be reproduced. But the measurement statistics cannot violate quantum predictions if quantum equilibrium is assumed. Therefore, quantum equilibrium precludes knowledge of the sub-ensemble sequence by the experimenters. The experimenters may assign subjective (Bayesian) probabilities to each possible sequence of sub-ensembles. Say the sequences are labelled by k, where k ∈{1,2,...W}. Then these subjective probabilities p(k) may be associated with a formal mathematical entropy

    S = -∑_k=1^W p(k) log_2 p(k)
 

This is the Shannon entropy of the distribution of sub-ensemble sequences over the N_0 runs. Note that this entropy is subjective in the sense that there is only one actual sequence for the experiment, but the experimenters have assigned probabilities p(k) to different sequences out of ignorance of the actual sequence.




Next, let the experimenters determine the setting mechanisms (γ_A and γ_B) for each run. Constraint (<ref>) implies that E = (γ_A, γ_B) for each run. Therefore, one of the probabilities p(k) then equals 1 and the rest are all 0. The associated entropy S = -1log_2 1 = 0, and the entropy has therefore decreased by

    Δ S =∑_k=1^W p(k) log_2 p(k)


As the experimenter at wing A (B) has to make no conscious effort to match γ_A (γ_B) values with the sub-ensemble sequence, it is fair to say that this entropy decrease is spontaneous (see Fig. 4). Note that Δ S = - H(k : {γ_A, γ_B}), where H(k : {γ_A, γ_B}) is the mutual information <cit.> between the variables k and {γ_A, γ_B} (the set of γ_A, γ_B values for N_0 runs). Note also that Δ S does not depend on p(λ|M_A, M_B) or p({γ_A, γ_B}). 


It is important to assume equation (<ref>) to derive this entropy decrease. Suppose that equation (<ref>) is not true. There are two possibilities in this case. First, λ is correlated to multiple setting mechanisms. In that case, the sub-ensembles and the associated entropy S cannot be defined. Second, λ is correlated to individual setting mechanisms, but these are not the ones chosen by the experimenters. In that case, the experimenters can assign probabilities p(k) to different sub-ensemble sequences as before. But, after choosing γ_A and γ_B for each run, the entropy S ≠ 0. 


We now evaluate the net spontaneous entropy drop for the simplest case that p(k)=1/W ∀ k. In this case, the net entropy drop can be thought of as the sum over N_0 runs of the individual entropy drop for each run. The net entropy drop is found to be 

    Δ S = -2N_0log_2 N


In terms of mutual information, equation (<ref>) implies that H(E : γ_A, γ_B) = 2log_2 N. Let us compare this with the mutual information between λ and the measurement settings for superdeterministic models of a Bell scenario for N=1. Ref. <cit.> reports a superdeterministic model which needs only ∼ 0.08 bits of mutual information between λ and the measurement settings. On the other hand, equation (<ref>) implies that for N = 16, H(E : γ_A, γ_B)  = 8 bits, which is ∼ 100 times the mutual information required between λ and the measurement settings for the aforementioned superdeterministic model. The extremely strong correlation between E and γ_A, γ_B arranged by the initial conditions is a conspiratorial feature of superdeterministic models of our scenario. In the limit N →∞, we have Δ S →∞ and in this sense we may say that the superdeterministic model becomes arbitrarily conspiratorial. 


As in the previous approach, nonlocal and retrocausal models are not conspiratorial according to this approach. This is because the variables {α}, γ_A and {β}, γ_B can be excluded from these models (see section <ref>), so that there are no sub-ensembles associated with different setting mechanisms. 




§ DISCUSSION
 
We have provided two different ways to quantify the often-alleged conspiratorial character of superdeterministic models, using intuitive suggestions from a previous article 𝒜. The first approach utilises the idea that different setting mechanisms lead to different measurement statistics in general for a nonequilibrium superdeterministic model. We have shown that superdeterministic models must be fine-tuned so that the measurement statistics depend on the measurement settings, but not on how these settings are chosen. No features of quantum mechanics, such as the Born rule, marginal-independence (formal no-signalling), non-contextuality etc. have been assumed in deriving this result. Further, we have quantified the fine tuning by defining an overhead fine-tuning parameter F. We calculated F for two different kinds of superdeterministic models of our scenario. We first considered superdeterministic models as usually proposed in the literature, with a single hidden-variables distribution p(λ|M_A, M_B). The fine tuning for such a model is drastic as F = 1 (corresponding to `completely fine tuned') for any N > 1. This result makes use of the condition Λ→∞ from Hardy's `excess-baggage' theorem <cit.>. We have also considered more general superdeterministic models of our scenario. For these general models, F < 1 for any N > 1. In the limit N →∞, F = 1. This result does not depend on the condition Λ→∞.


It is useful to discuss some possible misunderstandings about this result. One might argue that, in a nonequilibrium framework, an enormous amount of fine-tuning is inevitable anyway just to restrict outcomes to the Born rule. A bit more fine-tuning is therefore perhaps nothing to be concerned about. A counter example to such an argument would be the pilot-wave theory, where no fine tuning is necessary to reproduce the Born rule, even when nonequilibrium is allowed. This is because a nonequilibrium distribution in pilot-wave theory dynamically relaxes to the equilibrium distribution over time, given appropriate assumptions about the initial distribution <cit.>. Note that equilibration as an explanation of fine tuning is noted in the causal modelling literature as well <cit.>. Another possibility is that one might argue that an un-fine-tuned (F=0) classical universe will have a high-entropy past. The fine-tuning argument applied to our universe would therefore prove that a low-entropy past is very unlikely, disproving the second law of thermodynamics. However, there remain ambiguities about whether the entropy of the whole universe is a well-defined concept because of the role of gravity <cit.>. In the early universe, this problem is severe because gravitational effects cannot be ignored. Therefore, it is not clear at present whether an explanation of thermodynamics in terms of a low-entropy past of the universe can be given.


The parameter F quantifies how special the hidden-variables distribution has to be so that the measurement statistics depend only on the measurement settings. Clearly, the notion of `special' is meaningful only when there are multiple possible distributions, that is, if nonequilibrium is allowed (at least in principle). Those who consider nonequilibrium distributions to be unmotivated may find this approach to be artificial. We have, therefore, provided a second way to quantify the conspiracy, which does not use nonequilibrium distributions. 


The second approach uses another idea from 𝒜 that the violation of marginal independence (formal no-signalling) in superdeterministic models in nonequilibrium does not imply actual signalling. This leads to the intuitively conspiratorial situation where two experimenters can use marginal dependence as a practical signalling procedure without actually sending signals. In this case, the entire sequence of messages exchanged between the experimenters is interpreted as a statistical coincidence. Analogous to this, for a superdeterministic model of our scenario restricted to equilibrium, there occurs a one-to-one correspondence between the experimenters' choice of setting mechanisms and the sub-ensemble values for all runs. But this correspondence is, again, only a sequence of statistical coincidences. This appears conspiratorial as it shows that the initial conditions in superdeterministic models of our scenario need to arrange extremely strong correlations. Note that there have been several studies of the amount of correlation required between λ and the measurement settings in superdeterministic models <cit.>. A low correlation value might be interpreted in favour of superdeterministic models on grounds of efficiency (as has been suggested for retrocausal models in ref. <cit.>). Our argument shows that this would be incorrect: there has to be an extremely strong correlation (one-to-one correspondence) between the experimenters' choices and the sub-ensemble values in our scenario regardless of the strength of correlation between λ and the measurement settings. We quantify the correlation as a formal entropy drop Δ S defined at the hidden-variables level. We show that Δ S →∞ logarithmically fast with N, and in this sense we may say that the equilibrium superdeterministic model becomes arbitrarily conspiratorial as N →∞. 


Nonlocal and retrocausal models of our scenario are non-conspiratorial by definition according to either approach, as the variables related to the different setting mechanisms and experimenters' choices can be eliminated from the model description. Note that nonlocal, retrocausal and superdeterministic models are all conspiratorial in the sense that they require fine-tuning in order to satisfy marginal independence (formal no-signalling) <cit.>, unless a dynamical explanation of finetuning is given <cit.>. Our results here are concerned only with the notion of superdeterministic conspiracy (see 𝒜).


We have quantitatively proven that superdeterministic models are conspiratorial in two different ways. What are the possible options for a future superdeterminist? Let us consider the possible options for circumventing our two arguments separately. Consider first the fine-tuning argument. One option to evade fine tuning might be to eschew the possibility of quantum nonequilibrium. This will involve abandoning the distinction between initial conditions and laws, which is a central principle in scientific theories. Palmer seems to advocate such an approach <cit.>. However, his model still appears to assume an equilibrium distribution over certain parameters, a distribution which could in principle be different[See ref. <cit.> for a detailed discussion of Palmer's proposal in a hidden-variables framework.]. Another option might be to build a superdeterministic model with a dynamical relaxation mechanism such that an arbitrary distribution that does not satisfy equation (<ref>) evolves over time to a distribution that satisfies it.[Given appropriate assumptions about the initial distribution, as in classical statistical mechanics <cit.>.] For example, dynamical relaxation to the equilibrium distribution is a natural consequence of the evolution law in pilot-wave theory <cit.>. In such a superdeterministic model, however, small deviations from equilibrium may occur. This is because relaxation to the equilibrium distribution may get delayed due to various factors <cit.>. For quantum systems trapped in nonequilibrium, the superdeterministic model will predict the violation of condition (<ref>). One may use this to experimentally test such a superdeterministic model in the nonequilibrium regime. Let us now consider the spontaneous entropy drop argument. This approach does not depend on quantum nonequilibrium, but it is applicable only to models that satisfy condition (<ref>). Therefore, a superdeterministic model that violates condition (<ref>) can circumvent this argument. However, in such a model λ will not only be correlated with the physical variables that determine the measurement settings, but also with other physical variables that might have been used by the experimenters to choose the settings. These other variables could in principle be any variable from the entire observable universe. A superdeterministic model that violates (<ref>) would therefore appear to exchange one kind of conspiracy for another. 


We conclude that, at least as they are currently understood, superdeterministic models are conspiratorial. On this basis it seems fair to conclude that superdeterminism, as a possible explanation of the Bell correlations, is scientifically unattractive.

We are grateful to the anonymous referees for helpful comments.

bhak


