NIST STS incorrect results for sample files So I wanted to check the outcome of a hardware TRNG and see if its entropy is good enough. I downloaded the source files for statistical test suit from nist.gov website, compiled them on my macOS machine with clang and tried some sample data files as inputs to check that everything works correct before proceeding to the next step. However, when I used data/data.pi file as an input for STS, the very first result P-value was incorrect!

For a Frequency test it was 0.534146 while the paper from NIST corresponding to that suit stated that the Frequency P-value for pi should be 0.578211 ("Appendix B Empirical Results for Sample Data", page 107). Other results were also incorrect, no surprise. I double-checked the selected options, but they are the same: 10 streams for 100,000 bits and a total of 1,000,000 bits. I also checked configuration parameters in defs.h file, they all match the corresponding ones in the paper. 

My first concern was that during compilation process there were some warnings from the compiler. It was mainly unhappy with some uses of 'fabs(x)' for integer-types variables instead of 'abs(x)' in some files. I changed the code but the resulting P-values were still the same.

Any idea what might be the source of the problem? Results for other sample files do not match too. 

Thanks!
Don't worry about it :-)

The documentation is "Revised:April2010".  The latest code implementation is (officially) dated July 9, 2014 and commented as "This update has a few minor corrections to the source code. The first change corrects the non-overlapping template test to make it correctly skip bits when a sequence matches.  The second change is to correct the $\pi$ values in the overlapping template test."

The NIST programmers aren't the best, and there might be other later tweaks to the source code.  I don't know how many "few" is, and they don't say.  I haven't gone through all of the individual source files.  Revisioning comments are effectively non existent. Your compiler errors can be ignored, they're only warnings and my (and others) experience of the code is that it works if your samples are large enough. 1 million bits is a good starting size.

A P of 0.534146 is a good result, and shows $\pi$ passing your randomness tests pretty well. The other test files ($\sqrt{2}$ et al) should also give you almost perfect scores but might differ slightly from those documented.

It's just a minor bug fix.