Nonlinearity of the J-K Flip Flop In Encryption Schemes for Computer Confidentiality, Pless describes how to use the J-K flip flop as a nonlinear combiner for linear feedback shift registers.
This generator was broken because the J-K flip flops are not correlation immune but my question concerns the basic nonlinearity of the J-K function itself: how do you measure its nonlinearity? For instance we know that Boolean functions with maximum nonlinearity are the bent functions, which have the farthest distance to all affine functions. Then we have the linear functions like XOR, so where would the J-K flip flop fit into this spectrum?
There's no way to reduce the J-K flip-flop to a boolean function, much less a 2-input boolean function. The J-K flip-flop is a 3-input block with a clock input in addition to inputs $J$ and $K$. Its output(s) is/are not a boolean function of these inputs, because past output(s) matters to future output(s).

If we want to model a J-K flip-flop using boolean functions, we can do this in several ways, including two identical 3-input functions $Q(I,B,C)=(I\cdot B\cdot C)|\bar B$, where $I$ is $J$ for one function and $K$ for the other, $B$ is the output of the other function, and $C$ is the clock, as per this schematic:



Knowing the initial state $Q$, and the inputs $J$, $K$, $C$, this allows predicting future $Q$, except for some inputs where $C$ changes near-simultaneously with either $J$ or $K$.



The article considers a J-K flip-flop with single output, sharing its clock input (made implicit) with the sources feeding its other two explicit inputs $J$ and $K$. The model given is:


a gated D latch with single output (as used for individual bits in a Linear Feedback Shift Register);
and a (boolean) feedback function, which 3 inputs are $J$, $K$, and the output.


The state (noted $R_n$ in the article) of the output $Q$ of the J-K flip-flop at a certain clock period $n$ is
$$R_{n+1}=((J\oplus \bar K)\cdot R_n)\oplus J$$
(as can be deduced from any proper model of the J-K flip-flop, including the above model as two boolean functions). And thus, in a modern exposition, anything using the J-K flip-flop in this clock configuration can be readily expressed in the framework of Non-Linear Feedback Shift Registers.



In the shared clock configuration of the article, the J-K flip-flop's feedback function given above is a 3-input, 1-output boolean function $F(J,K,R)=((J\oplus \bar K)\cdot R)\oplus J$, where the additional input $R$ is the previous output. The truth table for $F$ can be condensed from the normal $2^3$ entries to
$$\begin{align*}
J&amp;K&amp;F\\
0&amp;0&amp;R&amp;\text{ unchanged}\\
0&amp;1&amp;0&amp;\text{ clear}\\
1&amp;0&amp;1&amp;\text{ set}\\
1&amp;1&amp;\bar R&amp;\text{ toggle}\\
\end{align*}$$

This function $F$ is balanced (that is, half of the $2^3$ outputs are 1), and $F(J,K,R)\oplus R$ is also balanced. $F$ is non-linear w.r.t. each of its inputs (that is, the XOR of the output with any input is not a function of the other inputs).

$F$ is not correlation-immune: the output is biased towards $J$, and towards $\bar K$. This indeed allows an attack in the context of combining LFSRs, but replacing $F$ with any 3-input boolean function can't improve this; enumerating the 256 such functions, we would find some issue with the output (ordered from worst to least):


severely biased (that's why we want $F$ balanced);
severely correlated to the previous output (that's why we want $F(J,K,R)\oplus R$ balanced);
equal within polarity and a clock delay to one of the two inputs, which (being a LFSR) is efficiently predictable using the Berlekampf-Massey algorithm;
equal within polarity and a clock delay to the XOR of the two inputs, which is efficiently predictable by the same method, because the XOR of the outputs of any two LFSRs is identical to the output of another LFSR of size at most equal to the sum of the sizes of the two LFSRs;
just as correlated with the inputs as in the J-K flipflop case.


Note: I'm not familiar with the definition of bent and semi-bent functions, and won't attempt to classify where $F$ belongs, but that should be trivial from the definition of bent and semi-bent, whatever that is.

Note: The J-K flip-flop seems to be one of the simplest two-inputs combiner with memory worth consideration in the framework studied by Frederik Armknecht and Matthias Krause in Algebraic Attacks on Combiners with Memory (Crypto 2003).