title: Unexplainable lack of performance improvement using RxJava Observables in Web Applications
body: <p>I am performing some tests to evaluate if there is a real advantage in using reactive API's based on Observables, instead of the blocking traditional ones.</p>
  
  <p>The whole example is <a href="https://github.com/codependent/spring-nio-rest">available on Githug</a></p>
  
  <p>Surprisingly the results show that the thoughput results are:  </p>
  
  <ul>
  <li><p><strong>The best</strong>: REST Services that return a <code>Callable</code>/<code>DeferredResult</code> that wraps the blocking operations.</p></li>
  <li><p><strong>Not that bad</strong>: Blocking REST Services.</p></li>
  <li><p><strong>The worst</strong>: REST Services that return a DeferredResult whose result is set by a <strong>RxJava Observable</strong>.</p></li>
  </ul>
  
  <p>This is my Spring WebApp:</p>
  
  <p><strong>Application</strong>:</p>
  
  <pre><code>@SpringBootApplication
  public class SpringNioRestApplication {
  
     @Bean
      public ThreadPoolTaskExecutor executor(){
          ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
          executor.setCorePoolSize(10);
          executor.setMaxPoolSize(20);
          return executor;
      }
  
      public static void main(String[] args) {
          SpringApplication.run(SpringNioRestApplication.class, args);
      }
  }
  </code></pre>
  
  <p><strong>SyncController</strong>:</p>
  
  <pre><code>@RestController("SyncRestController")
  @Api(value="", description="Synchronous data controller")
  public class SyncRestController {
  
      @Autowired
      private DataService dataService;
  
      @RequestMapping(value="/sync/data", method=RequestMethod.GET, produces="application/json")
      @ApiOperation(value = "Gets data", notes="Gets data synchronously")
      @ApiResponses(value={@ApiResponse(code=200, message="OK")})
      public List&lt;Data&gt; getData(){
          return dataService.loadData();
      }
  }
  </code></pre>
  
  <p><strong>AsyncController</strong>: With both raw Callable and Observable endpoints</p>
  
  <pre><code>@RestController
  @Api(value="", description="Synchronous data controller")
  public class AsyncRestController {
  
      @Autowired
      private DataService dataService;
  
      private Scheduler scheduler;
  
      @Autowired
      private TaskExecutor executor;
  
       @PostConstruct
      protected void initializeScheduler(){
          scheduler = Schedulers.from(executor);
      }
  
      @RequestMapping(value="/async/data", method=RequestMethod.GET, produces="application/json")
      @ApiOperation(value = "Gets data", notes="Gets data asynchronously")
      @ApiResponses(value={@ApiResponse(code=200, message="OK")})
      public Callable&lt;List&lt;Data&gt;&gt; getData(){
          return ( () -&gt; {return dataService.loadData();} );
      }
  
      @RequestMapping(value="/observable/data", method=RequestMethod.GET, produces="application/json")
       @ApiOperation(value = "Gets data through Observable", notes="Gets data asynchronously through Observable")
       @ApiResponses(value={@ApiResponse(code=200, message="OK")})
       public DeferredResult&lt;List&lt;Data&gt;&gt; getDataObservable(){
           DeferredResult&lt;List&lt;Data&gt;&gt; dr = new DeferredResult&lt;List&lt;Data&gt;&gt;();
           Observable&lt;List&lt;Data&gt;&gt; dataObservable = dataService.loadDataObservable();
           dataObservable.subscribeOn(scheduler).subscribe( dr::setResult, dr::setErrorResult);
           return dr;
       }
  }
  </code></pre>
  
  <p><strong>DataServiceImpl</strong></p>
  
  <pre><code>@Service
  public class DataServiceImpl implements DataService{
  
      @Override
      public List&lt;Data&gt; loadData() {
          return generateData();
      }
  
      @Override
      public Observable&lt;List&lt;Data&gt;&gt; loadDataObservable() {
          return Observable.create( s -&gt; {
              List&lt;Data&gt; dataList = generateData();
              s.onNext(dataList);
              s.onCompleted();
          });
      }
  
      private List&lt;Data&gt; generateData(){
          List&lt;Data&gt; dataList = new ArrayList&lt;Data&gt;();
          for (int i = 0; i &lt; 20; i++) {
              Data data = new Data("key"+i, "value"+i);
              dataList.add(data);
          }
          //Processing time simulation
          try {
              Thread.sleep(500);
          } catch (InterruptedException e) {
              e.printStackTrace();
          }
          return dataList;
      }
  }
  </code></pre>
  
  <p>I have set a <code>Thread.sleep(500)</code> delay to increase the service response time.</p>
  
  <p>There results from the load tests are:</p>
  
  <p><strong>Async with Callable</strong>: 700 rps, no errors</p>
  
  <pre><code>&gt;&gt;loadtest -c 15 -t 60 --rps 700 http://localhost:8080/async/data    
  ...
  Requests: 0, requests per second: 0, mean latency: 0 ms
  Requests: 2839, requests per second: 568, mean latency: 500 ms
  Requests: 6337, requests per second: 700, mean latency: 500 ms
  Requests: 9836, requests per second: 700, mean latency: 500 ms
  ...
  Completed requests:  41337
  Total errors:        0
  Total time:          60.002348360999996 s
  Requests per second: 689
  Total time:          60.002348360999996 s
  </code></pre>
  
  <p><strong>Blocking</strong>: around 404 rps but produces errors </p>
  
  <pre><code>&gt;&gt;loadtest -c 15 -t 60 --rps 700 http://localhost:8080/sync/data    
  ...
  Requests: 7683, requests per second: 400, mean latency: 7420 ms
  Requests: 9683, requests per second: 400, mean latency: 9570 ms
  Requests: 11680, requests per second: 399, mean latency: 11720 ms
  Requests: 13699, requests per second: 404, mean latency: 13760 ms
  ...
  Percentage of the requests served within a certain time
    50%      8868 ms
    90%      22434 ms
    95%      24103 ms
    99%      25351 ms
   100%      26055 ms (longest request)
  
   100%      26055 ms (longest request)
  
     -1:   7559 errors
  Requests: 31193, requests per second: 689, mean latency: 14350 ms
  Errors: 1534, accumulated errors: 7559, 24.2% of total requests
  </code></pre>
  
  <p><strong>Async with Observable</strong>: <strong>not more than 20 rps, and gets errors sooner</strong></p>
  
  <pre><code>&gt;&gt;loadtest -c 15 -t 60 --rps 700 http://localhost:8080/observable/data
  Requests: 0, requests per second: 0, mean latency: 0 ms
  Requests: 90, requests per second: 18, mean latency: 2250 ms
  Requests: 187, requests per second: 20, mean latency: 6770 ms
  Requests: 265, requests per second: 16, mean latency: 11870 ms
  Requests: 2872, requests per second: 521, mean latency: 1560 ms
  Errors: 2518, accumulated errors: 2518, 87.7% of total requests
  Requests: 6373, requests per second: 700, mean latency: 1590 ms
  Errors: 3401, accumulated errors: 5919, 92.9% of total requests 
  </code></pre>
  
  <p>The Observable executes with a corePoolSize of 10, but increasing it to 50 didn't improve anything either.</p>
  
  <p>What could be the explanation?</p>
  
  <p><strong>UPDATE</strong>: As suggested by akarnokd I made the following changes. Moved from Object.create to Object.fromCallable in the service and reused the Scheduler in the controller, but still I get the same results.</p>