How to select "good" bits which pass NIST statistical tests x% of the time? I am trying to implement an algorithm where the first step is to select "good" bits using the NIST test suite. In particular, I have $k$ streams with $n$ bits each. The original paper Toward Sensor-Based Random Number Generation for Mobile and IoT Devices (PDF) states


  For a given bit to be ‘good’, it must pass at least 3 of the NIST tests at least 75% of the time. 


What would be the best way to go about implementing this?

The NIST tests which I am using are the frequency test, frequency test within a block, runs test, longest run of ones within a block, DFT test, binary matrix rank test, and approximate entropy test.
From the question:


  I have $k$ streams with $n$ bits each. (..)
  The original paper (..) states "For a given bit to be ‘good’, it must pass at least 3 of the NIST tests at least 75% of the time"


I read "given bit" as meaning "any particular bit index in each of the $k$ streams"; and "of the time" as meaning "of each test run performed for bits at this bit index" (among possibly many such tests because $k$ exceeds the number of bits a given NIST test consumes). Thus the literal implementation could be:


for $i$ from $0$ to $n-1$ (each of $n$ bit index in a sequence)


$p\gets0$ (the number of tests passed for bit $i$) 
for each of the NIST tests considered (out of 15 at the last count)


let $b$ be the number of bits required by the particular NIST test (e.g. 100 for the Frequency (Monobit) Test; 1000 for the DFT test)
$f\gets\lfloor k/4b\rfloor$ (the maximum number of times the test is allowed to fail)
for $j$ from $0$ to $\lfloor k/b\rfloor-1$ (each group of $b$ streams) 


perform the test on the $b$-bit sequence obtained by keeping bit $i$ of sequences $j\cdot b$ to $(j+1)\cdot b-1$ (numbering sequences and bits starting from $0$)
if the test fails


$f\gets f-1$
(optional: fail quickly) if $f&lt;0$


break from the loop for $j$



if $f\ge0$ (the particular test passed at least 75% of the time)


$p\gets p+1$
(optional: succeed quickly) if $p\ge 3$


break from the loop for NIST tests



if $p\ge 3$ (at least 3 of the NIST tests passed at least 75% of the time)


that bit $i$ pass the original paper's criteria; remember to use this bit in the future.




It is possible to adjust $b$ for each test as a function of $k$ so as to reduce the proportion $k\bmod b$ out of $k$ bits that are not tested, or splitting the $k$ bits almost evenly across sequences tested (within 1 for tests with no constraint on $b$).

NIST tests with $b&gt;k$ succeed for 0 out or 0 time they are run, and accordingly the "at least 75% of the time" criteria is met; thus a test that does not run still accounts in the minimum of 3 tests that pass; I'm among those thinking this is a telltale sign that the criteria is profoundly unsound. Main problem really are:


As far as we can tell from the question, NIST tests are run unmodified, thus with low $p$-value (like 1%), making the 25% failure rate tolerated seemingly unjustifiable.
When any test consistently fails, the bit sequence has been distinguished from uniformly independent bits, yet it is sometime kept; this means we are not using the tests for what they are designed to do: test sequences against the null hypothesis that they are uniformly independent bits.
Treating all the tests equals (when they are designed to detect very specific defects) is a telltale sign of an absence of method.


My algorithm is only the literal implementation of what the paper prescribes. This shall not be interpreted as a recommendation; rather, the opposite: the rationale of the prescription evades me. Advice only binds who trusts it.