title: Flakiness in tests on Android using LiveData, RxJava/RxKotlin and Spek
body: <p>Setup:</p>
  
  <p>In our project (at work - I cannot post real code), we have implemented clean MVVM. Views communicate with ViewModels via LiveData. ViewModel hosts two kinds of use cases: 'action use cases' to do something, and 'state updater use cases'. Backward communication is asynchronous (in terms of action reaction). It's not like an API call where you get the result from the call. It's BLE, so after writing the characteristic there will be a notification characteristic we listen to. So we use a lot of Rx to update the state. It's in Kotlin.</p>
  
  <p>ViewModel:</p>
  
  <pre><code>@PerFragment
  class SomeViewModel @Inject constructor(private val someActionUseCase: SomeActionUseCase,
                                          someUpdateStateUseCase: SomeUpdateStateUseCase) : ViewModel() {
  
      private val someState = MutableLiveData&lt;SomeState&gt;()
  
      private val stateSubscription: Disposable
  
      // region Lifecycle
      init {
          stateSubscription = someUpdateStateUseCase.state()
                  .subscribeIoObserveMain() // extension function
                  .subscribe { newState -&gt;
                      someState.value = newState
                  })
      }
  
      override fun onCleared() {
          stateSubscription.dispose()
  
          super.onCleared()
      }
      // endregion
  
      // region Public Functions
      fun someState() = someState
  
      fun someAction(someValue: Boolean) {
          val someNewValue = if (someValue) "This" else "That"
  
          someActionUseCase.someAction(someNewValue)
      }
      // endregion
  }
  </code></pre>
  
  <p>Update state use case:</p>
  
  <pre><code>@Singleton
  class UpdateSomeStateUseCase @Inject constructor(
              private var state: SomeState = initialState) {
  
      private val statePublisher: PublishProcessor&lt;SomeState&gt; = 
              PublishProcessor.create()
  
      fun update(state: SomeState) {
          this.state = state
  
          statePublisher.onNext(state)
      }
  
      fun state(): Observable&lt;SomeState&gt; = statePublisher.toObservable()
                                                         .startWith(state)
  }
  </code></pre>
  
  <p>We are using Spek for unit tests.</p>
  
  <pre><code>@RunWith(JUnitPlatform::class)
  class SomeViewModelTest : SubjectSpek&lt;SomeViewModel&gt;({
  
      setRxSchedulersTrampolineOnMain()
  
      var mockSomeActionUseCase = mock&lt;SomeActionUseCase&gt;()
      var mockSomeUpdateStateUseCase = mock&lt;SomeUpdateStateUseCase&gt;()
  
      var liveState = MutableLiveData&lt;SomeState&gt;()
  
      val initialState = SomeState(initialValue)
      val newState = SomeState(newValue)
  
      val behaviorSubject = BehaviorSubject.createDefault(initialState)
  
      subject {
          mockSomeActionUseCase = mock()
          mockSomeUpdateStateUseCase = mock()
  
          whenever(mockSomeUpdateStateUseCase.state()).thenReturn(behaviorSubject)
  
          SomeViewModel(mockSomeActionUseCase, mockSomeUpdateStateUseCase).apply {
              liveState = state() as MutableLiveData&lt;SomeState&gt;
          }
      }
  
      beforeGroup { setTestRxAndLiveData() }
      afterGroup { resetTestRxAndLiveData() }
  
      context("some screen") {
          given("the action to open the screen") {
              on("screen opened") {
                  subject
                  behaviorSubject.startWith(initialState)
  
                  it("displays the initial state") {
                      assertEquals(liveState.value, initialState)
                  }
              }
          }
  
          given("some setup") {
              on("some action") {
                  it("does something") {
                      subject.doSomething(someValue)
  
                      verify(mockSomeUpdateStateUseCase).someAction(someOtherValue)
                  }
              }
  
              on("action updating the state") {
                  it("displays new state") {
                      behaviorSubject.onNext(newState)
  
                      assertEquals(liveState.value, newState)
                  }
              }
          }
      }
  }
  </code></pre>
  
  <p>At first we were using an Observable instead of the BehaviorSubject:</p>
  
  <pre><code>var observable = Observable.just(initialState)
  ...
  whenever(mockSomeUpdateStateUseCase.state()).thenReturn(observable)
  ...
  observable = Observable.just(newState)
  assertEquals(liveState.value, newState)
  </code></pre>
  
  <p>instead of the:</p>
  
  <pre><code>val behaviorSubject = BehaviorSubject.createDefault(initialState)
  ...
  whenever(mockSomeUpdateStateUseCase.state()).thenReturn(behaviorSubject)
  ...
  behaviorSubject.onNext(newState)
  assertEquals(liveState.value, newState)
  </code></pre>
  
  <p>but the unit test were being flaky. Mostly they would pass (always when ran in isolation), but sometime they would fail when running the whole suit. Thinking it is to do with asynchronous nature of the Rx we moved to BehaviourSubject to be able to control when the onNext() happens. Test are now passing when we run them from AndroidStudio on the local machine, but they are still flaky on the build machine. Restarting the build often makes them pass. </p>
  
  <p>The tests which fail are always the ones where we assert the value of LiveData. So the suspects are LiveData, Rx, Spek or their combination.</p>
  
  <p><strong>Question:</strong> Did anyone have similar experiences writing unit tests with LiveData, using Spek or maybe Rx, and did you find ways to write them which solve these flakiness issues?</p>
  
  <p>....................</p>
  
  <p>Helper and extension functions used:</p>
  
  <pre><code>fun instantTaskExecutorRuleStart() =
          ArchTaskExecutor.getInstance().setDelegate(object : TaskExecutor() {
              override fun executeOnDiskIO(runnable: Runnable) {
                  runnable.run()
              }
  
              override fun isMainThread(): Boolean {
                  return true
              }
  
              override fun postToMainThread(runnable: Runnable) {
                  runnable.run()
              }
          })
  
  fun instantTaskExecutorRuleFinish() = ArchTaskExecutor.getInstance().setDelegate(null)
  
  fun setRxSchedulersTrampolineOnMain() = RxAndroidPlugins.setInitMainThreadSchedulerHandler { Schedulers.trampoline() }
  
  fun setTestRxAndLiveData() {
      setRxSchedulersTrampolineOnMain()
      instantTaskExecutorRuleStart()
  }
  
  fun resetTestRxAndLiveData() {
      RxAndroidPlugins.reset()
      instantTaskExecutorRuleFinish()
  }
  
  fun &lt;T&gt; Observable&lt;T&gt;.subscribeIoObserveMain(): Observable&lt;T&gt; =
          subscribeOnIoThread().observeOnMainThread()
  
  fun &lt;T&gt; Observable&lt;T&gt;.subscribeOnIoThread(): Observable&lt;T&gt; = subscribeOn(Schedulers.io())
  
  fun &lt;T&gt; Observable&lt;T&gt;.observeOnMainThread(): Observable&lt;T&gt; =
          observeOn(AndroidSchedulers.mainThread())
  </code></pre>