title: Web scraping client certificate issue WinHttp - excel VBA
body: <p>So, I'm trying to scrape the following public site using vba and the <code>WinHttp</code> library:</p>
  <p><a href="https://auctions.seecao.com/DAILY_AUCTION_LIST" rel="nofollow noreferrer">https://auctions.seecao.com/DAILY_AUCTION_LIST</a></p>
  <p>Having examined the network traffic when the &quot;Show Data&quot; button is clicked, I came up with my code:</p>
  <pre><code>Sub test()
  Dim border As String
  Dim req As New WinHttpRequest
  Dim url As String
  Dim reqBodyObj As Object, respObj As Object, auction As Object
  Dim reqBodyStr As String
  Dim deliveryDay As Date
  url = &quot;https://auctions.seecao.com/api/DailyAuction/GetDailyAuctionList&quot;
  deliveryDay = Date
  border = &quot;ALME&quot;
  Set reqBodyObj = JsonConverter.ParseJson(&quot;{&quot;&quot;parameters&quot;&quot;:{&quot;&quot;dayFrom&quot;&quot;:&quot;&quot;2021-04-01&quot;&quot;,&quot;&quot;dayTill&quot;&quot;:&quot;&quot;2021-04-01&quot;&quot;,&quot;&quot;auctionState&quot;&quot;:[0,3,4,5,6,7,9]}}&quot;)
  reqBodyObj(&quot;parameters&quot;)(&quot;dayFrom&quot;) = Format(deliveryDay + 1, &quot;yyyy-mm-dd&quot;)
  reqBodyObj(&quot;parameters&quot;)(&quot;dayTill&quot;) = Format(deliveryDay + 1, &quot;yyyy-mm-dd&quot;)
  reqBodyStr = JsonConverter.ConvertToJson(reqBodyObj)
  
  With req
      .Open &quot;POST&quot;, url, False
      .setRequestHeader &quot;Content-Type&quot;, &quot;application/json&quot;
      .Option(WinHttpRequestOption_SslErrorIgnoreFlags) = 256 '=0x0100 =ignore &quot;Unknown certification authority (CA) or untrusted root&quot; error refer to: https://docs.microsoft.com/en-us/windows/win32/winhttp/winhttprequestoption
      .send reqBodyStr
      Debug.Print .responseText
      Set respObj = JsonConverter.ParseJson(.responseText)
  End With
  
  For Each auction In respObj(&quot;dailyAuctionListData&quot;)(&quot;rows&quot;)
      If auction(&quot;columns&quot;)(&quot;auctionName&quot;) Like border &amp; &quot;*&quot; Then
          Debug.Print auction(&quot;columns&quot;)(&quot;id&quot;)
      End If
  Next auction
  End Sub
  </code></pre>
  <p>It's probably worth noting that the first time you visit the site via a browser you'll get a warning saying that the server's certificate is not trusted and you'll have to add an exception to visit it:</p>
  <p><a href="https://i.stack.imgur.com/WW90L.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/WW90L.jpg" alt="enter image description here" /></a></p>
  <p>To overcome this I used this <code>.Option(WinHttpRequestOption_SslErrorIgnoreFlags) = 256</code> to ignore the errors.</p>
  <p><strong>Everything worked fine</strong> for about a month. Today however I started getting the following error:</p>
  <blockquote>
  <p>Run-time error '-2147012711 (80072f99)': No credentials were available
  in the client certificate.</p>
  </blockquote>
  <p>So I suppose that the website started requesting a client certificate?</p>
  <p>Requesting the data through a browser works normally without the need of any credentials and the <code>POST</code> request that is being sent under the hood seems to be the same as it used to be. Editing and resending the request via the browser's developer tools also works without any problems.</p>
  <p>So my question is, what is going on here? Is the site asking for a client certificate?</p>
  <p>And if so, which client certificate does Firefox use under the hood?</p>
  <p>Does the browser use some kind of a default certificate for these cases?</p>
  <p>Why am I asked for credentials when I run the code when clearly they are not needed when I visit the page?</p>
  <p>Finally, when I try to specify a random certificate like so :</p>
  <p><code>.SetClientCertificate &quot;NameOfCertificate&quot;</code></p>
  <p>the request is being sent and I get the following response:</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>&lt;html&gt;
  &lt;head&gt;&lt;title&gt;400 The SSL certificate error&lt;/title&gt;&lt;/head&gt;
  &lt;body&gt;
  &lt;center&gt;&lt;h1&gt;400 Bad Request&lt;/h1&gt;&lt;/center&gt;
  &lt;center&gt;The SSL certificate error&lt;/center&gt;
  &lt;hr&gt;&lt;center&gt;nginx/1.18.0&lt;/center&gt;
  &lt;/body&gt;
  &lt;/html&gt;</code></pre>
  </div>
  </div>
  </p>
  <p>Keep in mind that it's a public site accessible to everyone. No subscription needed.</p>
  <p><strong>EDIT</strong></p>
  <p>The same request works just fine with <code>Postman</code> after disabling SSL certificate verification. Is there a way to imitate this behavior in VBA?</p>