title: VBA - XMLHTTP and WinHttp request speed
body: <p>Below are declared variables for 3 requests which I implement in my macros. I listed libraries they use and their late bindings in comments:</p>
  
  <pre><code>Dim XMLHTTP As New MSXML2.XMLHTTP 'Microsoft XML, v6.0 'Set XMLHTTP = CreateObject("MSXML2.XMLHTTP.6.0")
  Dim ServerXMLHTTP As New MSXML2.ServerXMLHTTP 'Microsoft XML, v6.0 'Set ServerXMLHTTP = CreateObject("MSXML2.ServerXMLHTTP.6.0")
  Dim http As New WinHttpRequest 'Microsoft WinHttp Services, version 5.1 'Set http = CreateObject("WinHttp.WinHttpRequest.5.1")
  </code></pre>
  
  <p>I have a few old web scraping macros which used Internet Explorer automation. I wanted to clean coding and speed them up with these requests.</p>
  
  <p>Unfortunately what I have noticed, <code>MSXML2.ServerXMLHTTP</code> and <code>WinHttpRequest</code> are slower on online store's 20 products test (34 and 35 sec) than IE automation with pictures and active scripting off (24 sec)! <code>MSXML2.XMLHTTP</code> executes in 18 secs. I used to see situations when some out of these 3 requests are 2-3 times faster / slower than the other ones, so I always test which one performs best, but never before had any request lost to IE automation.</p>
  
  <p>The main page with results is below, it's all results on one page, 1500+ of them, so request takes some time (6500 pages if pasted to MS Word):</p>
  
  <p><a href="http://www.justbats.com/products/bat%20type~baseball/?sortBy=TotalSales%20Descending&amp;page=1&amp;size=2400" rel="nofollow noreferrer">www.justbats.com/products/bat type~baseball/?sortBy=TotalSales Descending&amp;page=1&amp;size=2400</a></p>
  
  <p>Then I open individual links from main result page:</p>
  
  <p><a href="http://www.justbats.com/product/2017-marucci-cat-7-bbcor-baseball-bat--mcbc7/24317/" rel="nofollow noreferrer">http://www.justbats.com/product/2017-marucci-cat-7-bbcor-baseball-bat--mcbc7/24317/</a></p>
  
  <p>I would like to know if these 3 requests are all options I have to get data from websites without browser automation. Also - how possibly browser automation can beat some of these requests?</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>I have tested the main result page with procedure provided in answer by Robin Mackenzie, clearing IE cache before running it. At least on this particular page, caching seemed to have no explicit gain, as subsequent requests yielded a similar result. IE had active scripting disabled and no images loading.</p>
  
  <pre><code>IE automation method, Document length: 7593346 chars, Processed in: 8 seconds
  
  WinHTTP method,  Document length: 7824059 chars, Processed in: 29 seconds
  
  XML HTTP method, Document length: 7830217 chars, Processed in: 4 seconds
  
  Server XML HTTP method, Document length: 7823958 chars, Processed in: 26 seconds
  
  URL download file method, Document length: 7830346 chars, Processed in: 7 seconds
  </code></pre>
  
  <p>Very surprising for me is the difference in amount of characters returned by these methods.</p>