updated to (7, 4). A bit is arithmetically coded with space proportional to its probability, either P(1) or P(0) = 1 − P(1). The probabilities are computed by weighted addition of the 0 and 1 counts: S0 = Σi wi n0i, S1 = Σi wi n1i, S = S0 + S1, P(0) = S0 / S, P(1) = S1 / S, where wi is the weight of the i-th model. Through PAQ3, the weights were fixed and set in an ad-hoc manner. (Order-n contexts had a weight of n2.) Beginning with PAQ4, the weights were adjusted adaptively in the direction that would reduce future errors in the same context set. If the bit to be coded is y, then the weight adjustment is: ni = n0i + n1i, error = y – P(1), wi ← wi + [(S n1i − S1 ni) / (S0 S1)] error. Neural-network mixing Beginning with PAQ7, each model outputs a prediction (instead of a pair of counts). These predictions are averaged in the logistic domain: xi = stretch(Pi(1)), P(1) = squash(Σi wi xi), where P(1) is the probability that the next bit will be a 1, Pi(1) is the probability estimated by the i-th model, and stretch(x) = ln(x / (1 − x)), squash(x) = 1 / (1 + e−x) (inverse of stretch). After each prediction, the model is updated by adjusting the weights to minimize coding cost: wi ← wi + η xi (y − P(1)), where η is the learning rate (typically 0.002 to 0.01), y is the predicted bit, and (y − P(1)) is the prediction error. The weight update algorithm differs from backpropagation in that the terms P(1)P(0) are dropped. This is because the goal of the neural network is to minimize coding cost, not root mean square error. Most versions of PAQ use a small context to select among sets of weights for the neural network. Some versions use multiple networks whose outputs are combined with one more network prior to the SSE stages. Furthermore, for each input prediction there may be several inputs which are nonlinear functions of Pi(1) in addition to stretch(P(1)). Context modeling Each model partitions the known bits of s into a set of contexts and maps each context to a bit history represented by an 8-bit state. In versions through PAQ6, the state represents a pair of counters (n0, n1). In PAQ7 and later versions