title: Time-dependent tests, how to ensure success?
body: <p>This is probably a timeless question from someone who has not taken the time yet to fully investigate the problem at hand yet, but here goes...</p>
  
  <p>The test is this:</p>
  
  <pre><code>@Test
  public void expiryWorksAsExpected()
      throws IOException, InterruptedException
  {
      final MessageSource source2 = mock(MessageSource.class);
      final MessageSource source3 = mock(MessageSource.class);
  
      when(loader.load(any(Locale.class)))
          .thenReturn(source)
          .thenReturn(source2)
          .thenReturn(source3);
  
      final MessageSourceProvider provider = builder.setLoader(loader)
          .setExpiryTime(10L, TimeUnit.MILLISECONDS).build();
  
      final MessageSource first = provider.getMessageSource(Locale.ROOT);
      TimeUnit.MILLISECONDS.sleep(50L);
      final MessageSource second = provider.getMessageSource(Locale.ROOT);
      TimeUnit.MILLISECONDS.sleep(50L);
      final MessageSource third = provider.getMessageSource(Locale.ROOT);
  
      verify(loader, times(3)).load(Locale.ROOT); // HERE
  
      assertSame(first, source);
      assertSame(second, source2);
      assertSame(third, source3);
  }
  </code></pre>
  
  <p>At the point marked <code>HERE</code> the test fails... From time to time (pun intended). But I don't understand why. So, I'll unroll the code here.</p>
  
  <p>First: <code>source</code> is a <code>mock(MessageSource.class)</code> (define test class wise), and the <code>MessageSource</code> code is as follows:</p>
  
  <pre><code>public interface MessageSource
  {
      String getKey(final String key);
  }
  </code></pre>
  
  <p>Second: <code>loader</code> is a <code>mock(MessageSourceLoader.class)</code>, which is:</p>
  
  <pre><code>public interface MessageSourceLoader
  {
      MessageSource load(final Locale locale)
          throws IOException;
  }
  </code></pre>
  
  <p>Third: <code>builder</code> is a <code>LoadingMessageSourceProvider.Builder</code>; full code below, with comments stripped (still a VERY long read, sorry for that):</p>
  
  <pre><code>@ThreadSafe
  public final class LoadingMessageSourceProvider
      implements MessageSourceProvider
  {
      private static final ThreadFactory THREAD_FACTORY = new ThreadFactory()
      {
          private final ThreadFactory factory = Executors.defaultThreadFactory();
  
          @Override
          public Thread newThread(final Runnable r)
          {
              final Thread ret = factory.newThread(r);
              ret.setDaemon(true);
              return ret;
          }
      };
  
      // From a custom API -- more details on demand
      private static final InternalBundle BUNDLE = InternalBundle.getInstance();
  
      private static final int NTHREADS = 3;
  
      private final ExecutorService service
          = Executors.newFixedThreadPool(NTHREADS, THREAD_FACTORY);
  
      private final MessageSourceLoader loader;
      private final MessageSource defaultSource;
  
      private final long timeoutDuration;
      private final TimeUnit timeoutUnit;
  
      private final AtomicBoolean expiryEnabled;
      private final long expiryDuration;
      private final TimeUnit expiryUnit;
  
      private final Map&lt;Locale, FutureTask&lt;MessageSource&gt;&gt; sources
          = new HashMap&lt;Locale, FutureTask&lt;MessageSource&gt;&gt;();
  
      private LoadingMessageSourceProvider(final Builder builder)
      {
          loader = builder.loader;
          defaultSource = builder.defaultSource;
  
          timeoutDuration = builder.timeoutDuration;
          timeoutUnit = builder.timeoutUnit;
  
          expiryDuration =  builder.expiryDuration;
          expiryUnit = builder.expiryUnit;
          expiryEnabled = new AtomicBoolean(expiryDuration == 0L);
      }
  
      public static Builder newBuilder()
      {
          return new Builder();
      }
  
      @Override
      public MessageSource getMessageSource(final Locale locale)
      {
          if (!expiryEnabled.getAndSet(true))
              setupExpiry(expiryDuration, expiryUnit);
  
          FutureTask&lt;MessageSource&gt; task;
  
          synchronized (sources) {
              task = sources.get(locale);
              if (task == null) {
                  task = loadingTask(locale);
                  sources.put(locale, task);
                  service.execute(task);
              }
          }
  
          try {
              final MessageSource source = task.get(timeoutDuration, timeoutUnit);
              return source == null ? defaultSource : source;
          } catch (InterruptedException ignored) {
              Thread.currentThread().interrupt();
              return defaultSource;
          } catch (ExecutionException ignored) {
              return defaultSource;
          } catch (TimeoutException ignored) {
              return defaultSource;
          } catch (CancellationException ignored) {
              return defaultSource;
          }
      }
  
      private FutureTask&lt;MessageSource&gt; loadingTask(final Locale locale)
      {
          return new FutureTask&lt;MessageSource&gt;(new Callable&lt;MessageSource&gt;()
          {
              @Override
              public MessageSource call()
                  throws IOException
              {
                  return loader.load(locale);
              }
          });
      }
  
      private void setupExpiry(final long duration, final TimeUnit unit)
      {
          final Runnable runnable = new Runnable()
          {
              @Override
              public void run()
              {
                  final List&lt;FutureTask&lt;MessageSource&gt;&gt; tasks;
                  synchronized (sources) {
                      tasks = new ArrayList&lt;FutureTask&lt;MessageSource&gt;&gt;(
                          sources.values());
                      sources.clear();
                  }
                  for (final FutureTask&lt;MessageSource&gt; task: tasks)
                      task.cancel(true);
              }
          };
          // Overkill?
          final ScheduledExecutorService scheduled
              = Executors.newScheduledThreadPool(1, THREAD_FACTORY);
          scheduled.scheduleAtFixedRate(runnable, duration, duration, unit);
      }
  
      public static final class Builder
      {
          private MessageSourceLoader loader;
          private MessageSource defaultSource;
          private long timeoutDuration = 1L;
          private TimeUnit timeoutUnit = TimeUnit.SECONDS;
          private long expiryDuration = 10L;
          private TimeUnit expiryUnit = TimeUnit.MINUTES;
  
          private Builder()
          {
          }
  
          public Builder setLoader(final MessageSourceLoader loader)
          {
              BUNDLE.checkNotNull(loader, "cfg.nullLoader");
              this.loader = loader;
              return this;
          }
  
          public Builder setDefaultSource(final MessageSource defaultSource)
          {
              BUNDLE.checkNotNull(defaultSource, "cfg.nullDefaultSource");
              this.defaultSource = defaultSource;
              return this;
          }
  
          public Builder setLoadTimeout(final long duration, final TimeUnit unit)
          {
              BUNDLE.checkArgument(duration &gt; 0L, "cfg.nonPositiveDuration");
              BUNDLE.checkNotNull(unit, "cfg.nullTimeUnit");
              timeoutDuration = duration;
              timeoutUnit = unit;
              return this;
          }
  
          public Builder setExpiryTime(final long duration, final TimeUnit unit)
          {
              BUNDLE.checkArgument(duration &gt; 0L, "cfg.nonPositiveDuration");
              BUNDLE.checkNotNull(unit, "cfg.nullTimeUnit");
              expiryDuration = duration;
              expiryUnit = unit;
              return this;
          }
  
          public Builder neverExpires()
          {
              expiryDuration = 0L;
              return this;
          }
  
          public MessageSourceProvider build()
          {
              BUNDLE.checkArgument(loader != null, "cfg.noLoader");
              return new LoadingMessageSourceProvider(this);
          }
      }
  }
  </code></pre>
  
  <p>Now, the problem: from time to time, I see the test fail; more specifically, at the line where it checks that the loader has been called exactly three times. Notwithstanding the fact that I have never seen the test fail before, that millisecond delays may be too short etc etc, I'd like to ensure that such a test runs and succeeds -- I want to test my logic. How do I do that without resorting to a 10 ms expiry time and an unreasonable (say, 2 seconds) sleep between fetches?</p>
  
  <p><strong>EDIT</strong> The purpose of the test is to validate that expiry times are obeyed; here I set a loader with a 10 ms expiry, and try to read from it a first time, then pause 50 ms, read a second time, then pause 50 ms, then read a third time; I want to ensure that the expiry works as I intend using mockito's chained <code>.thenReturn()</code> invocations</p>