Scraper for Redmart.com I've written a script which is scraping  a webpage encrypted within javascript. The scraper is extracting name, price and image link of certain products from that page. I used selenium in combination with vba to accomplish that. Hope I did it nicely.

Sub Redmart_scraping()
Dim driver As New WebDriver
Dim posts As Object, post As Object

Set driver = New WebDriver
driver.Start "Phantomjs", "https://redmart.com"
driver.get "/bakery"
Set posts = driver.FindElementsByXPath("//li[@class='productPreview  ']")
On Error Resume Next
For Each post In posts
    i = i + 1
    Cells(i, 1) = post.FindElementByXPath(".//h4/a").Text
    Cells(i, 2) = post.FindElementByXPath(".//span[@class='ProductPrice__price___3BmxE']").Text
    Cells(i, 3) = post.FindElementByXPath(".//img[contains(@class,'showImage')]").Attribute("src")
Next post
End Sub
I would improve on handling class attributes. They are multi-valued attributes per definition and your way of checking the class values in the XPath expressions can potentially cause problems.

For instance, @class='productPreview  ' check would fail if there will be one space less at the end of the value. Or, you would also match an element with class="showImageDescription" with the contains(@class,'showImage') check.

One way of dealing with that is to use this workaround which involves using concat().

I would though simply switch to CSS selectors which are generally faster, more concise and handle class attributes naturally:

Set posts = driver.FindElementsByCss("li.productPreview")
On Error Resume Next
For Each post In posts
    i = i + 1
    Cells(i, 1) = post.FindElementByCss("h4 &gt; a").Text
    Cells(i, 2) = post.FindElementByCss("span[class^=ProductPrice__price]").Text
    Cells(i, 3) = post.FindElementByCss("img.showImage").Attribute("src")
Next post


Note the span[class^=ProductPrice__price] selector - I don't think the ___3BmxE part of the class matters much and provides any relevant information - plus, it sounds like a good subject for a change. That's why I've switched to a partial match with "starts with" (^=).