title: Unit testing random numbers java
body: <p>My goal is to test whether a class sets one of its attributes to a random integer value. I found an algorithm for a chi square test online and decided to put it to use. I'm quite surprised by the result: the larger I make my sample size, the less likely the test seems to be to pass. I should say that I'm by no means a statistics expert (which might be self-evident by my asking this question) so I might be getting some things wrong here.</p>
  
  <p>Results of tests while variating only the final int <code>SIZE</code> (in <code>UserTest</code>). Every test ran 30 times:</p>
  
  <pre><code>SIZE    avg     results
  
  11      25.4    26, 25, 22, 24, 30
  
  20      25      26, 26, 24, 22, 27
  
  30      24      24, 22, 24, 26, 24
  
  100     19.4    17, 23, 20, 18, 19
  
  200     16.2    15, 18, 18, 15, 15
  
  1000    13.2    13, 13, 14, 13, 13
  
  10000   10      14, 7, 8, 10, 11
  </code></pre>
  
  <p>While it's not strictly necessary for me to have true randomness in this context, I'm still curious as to what the problem is. Is this a faulty algorithm in itself, my faulty use of it, a natural consequence of "making the test harder" (statistics noob, remember), or am I pushing up against the boundaries of Java's pseudorandom generator ?</p>
  
  <p>the domain class:</p>
  
  <pre><code>public class User
  {
    public static final int MINIT = 20;
    public static final int MAXIT = 50;
    private int iterations;
  
    public void setIterations()
    {
      Random random = new Random();
      setIterations(MINIT+random.nextInt(MAXIT-MINIT));
    }
  
    private void setIterations(int iterations) {
      this.iterations = iterations;
    }
  }
  </code></pre>
  
  <p>the test class:</p>
  
  <pre><code>public class UserTest {
  
    private User user = new User();
  
    @Test
    public void testRandomNumbers() {
      int results = 0;
      final int TIMES = 30;
      for(int i = 0; i &lt; TIMES; i++)
      {
        if (randomNumbersRun())
        {
          results++;
        }
      }
      System.out.println(results);
      Assert.assertTrue(results &gt;= TIMES * 80 / 100);
    }
  
    private boolean randomNumbersRun()
    {
      ArrayList&lt;Integer&gt; list = new ArrayList&lt;Integer&gt;();
      int r = User.MAXIT - User.MINIT;
      final int SIZE = 11;
      for (int i = 0; i &lt; r*SIZE; i++) {
        user.setIterations();
        list.add(user.getIterations());
      }
      return Statistics.isRandom(list, r);
    }
  }
  </code></pre>
  
  <p>The chi square algorithm:</p>
  
  <pre><code>  /**
     * source: http://en.wikibooks.org/wiki/Algorithm_Implementation/Pseudorandom_Numbers/Chi-Square_Test
     * changed parameter to ArrayList&lt;Number&gt; for generalization
     */
    public static boolean isRandom(ArrayList&lt;? extends Number&gt; randomNums, int r) {
      //According to Sedgewick: "This is valid if N is greater than about 10r"
      if (randomNums.size() &lt;= 10 * r) {
        return false;
      }
  
      //PART A: Get frequency of randoms
      Map&lt;Number, Integer&gt; ht = getFrequencies(randomNums);
  
      //PART B: Calculate chi-square - this approach is in Sedgewick
      double n_r = (double) randomNums.size() / r;
      double chiSquare = 0;
  
      for (int v : ht.values()) {
        double f = v - n_r;
        chiSquare += f * f;
      }
      chiSquare /= n_r;
  
      //PART C: According to Swdgewick: "The statistic should be within 2(r)^1/2 of r
      //This is valid if N is greater than about 10r"
      return Math.abs(chiSquare - r) &lt;= 2 * Math.sqrt(r);
    }
  
    /**
     * @param nums an array of integers
     * @return a Map, key being the number and value its frequency
     */
    private static Map&lt;Number, Integer&gt; getFrequencies(ArrayList&lt;? extends Number&gt; nums) {
      Map&lt;Number, Integer&gt; freqs = new HashMap&lt;Number, Integer&gt;();
  
      for (Number x : nums) {
        if (freqs.containsKey(x)) {
          freqs.put(x, freqs.get(x) + 1);
        } else {
          freqs.put(x, 1);
        }
      }
  
      return freqs;
    }
  }
  </code></pre>