title: How to efficiently do web scraping in Node.js?
body: <p>I am trying to scrape some data from a shopping site Express.com. Here's 1 of many products that contains <strong>image</strong>, <strong>price</strong>, <strong>title</strong>, <strong>color(s)</strong>.</p>
  
  <pre><code>  &lt;div class="cat-thu-product cat-thu-product-all item-1"&gt;
  &lt;div class="cat-thu-p-cont reg-thumb" id="p-50715" style="position: relative;"&gt;&lt;a href="/rocco-slim-fit-skinny-leg-corduroy-jean-50715-647/control/show/3/index.pro" onclick="var x=&amp;quot;.tl(&amp;quot;;s_objectID=&amp;quot;http://www.express.com/rocco-slim-fit-skinny-leg-corduroy-jean-50715-647/control/show/3/index.pro_1&amp;quot;;return this.s_oc?this.s_oc(e):true"&gt;&lt;img class="cat-thu-p-ima widget-app-quickview" src="http://t.express.com/com/scene7/s7d5/=/is/image/expressfashion/25_323_2516_900/i81?$dcat191$" alt="ROCCO SLIM FIT SKINNY LEG CORDUROY JEAN"&gt;&lt;/a&gt;&lt;a href="#jsLink"&gt;&lt;img id="widget-quickview-but" class="widget-ie6png glo-but-css-off2" src="/assets/images/but/cat/but-cat-quickview.png" alt="Express View" style="position: absolute; left: 50px;"&gt;&lt;/a&gt;&lt;/div&gt;
    &lt;ul&gt;
      &lt;li class="cat-cat-more-colors"&gt;
        &lt;div class="productId-50715"&gt;
          &lt;img class="js-swatchLinkQuickview" title="INK BLUE" src="http://t.express.com/com/scene7/s7d5/=/is/image/expressfashion/25_323_2516_900_s/i81?$swatch$" width="16" height="6" alt="INK BLUE"&gt;
          &lt;img class="js-swatchLinkQuickview" title="GRAPHITE" src="http://t.express.com/com/scene7/s7d5/=/is/image/expressfashion/25_323_2516_924_s/i81?$swatch$" width="16" height="6" alt="GRAPHITE"&gt;
          &lt;img class="js-swatchLinkQuickview" title="MERCURY GRAY" src="http://t.express.com/com/scene7/s7d5/=/is/image/expressfashion/25_323_2516_930_s/i81?$swatch$" width="16" height="6" alt="MERCURY GRAY"&gt;
          &lt;img class="js-swatchLinkQuickview" title="HARVARD RED" src="http://t.express.com/com/scene7/s7d5/=/is/image/expressfashion/25_323_2516_853_s/i81?$swatch$" width="16" height="6" alt="HARVARD RED"&gt;
        &lt;/div&gt;
      &lt;/li&gt;
      &lt;li class="cat-thu-name"&gt;&lt;a href="/rocco-slim-fit-skinny-leg-corduroy-jean-50715-647/control/show/3/index.pro" onclick="var x=&amp;quot;.tl(&amp;quot;;s_objectID=&amp;quot;http://www.express.com/rocco-slim-fit-skinny-leg-corduroy-jean-50715-647/control/show/3/index.pro_2&amp;quot;;return this.s_oc?this.s_oc(e):true"&gt;ROCCO SLIM FIT SKINNY LEG CORDUROY JEAN
      &lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;
        &lt;strong&gt;$88.00&lt;/strong&gt;
      &lt;/li&gt;
    &lt;li class="cat-thu-promo-text"&gt;&lt;font color="BLACK" style="font-weight:normal"&gt;Buy 1, Get 1 50% Off&lt;/font&gt;
    &lt;/li&gt;
  &lt;/ul&gt;
  </code></pre>
  
  <p></p>
  
  <p>The very naive and possibly error-prone approach I've done is to first to grab all prices, images, titles and colors:</p>
  
  <pre><code>var price_objects = $('.cat-thu-product li strong');
  var image_objects = $('.cat-thu-p-ima');
  var name_objects = $('.cat-thu-name a');
  var color_objects = $('.cat-cat-more-colors div');
  </code></pre>
  
  <p>Next, I populate arrays with the data from DOM extracted using <strong>jsdom</strong> or <strong>cheerio</strong> scraping libraries for node.js. (Cheerio in this case).</p>
  
  <pre><code>  // price info
    for (var i = 0; i &lt; price_objects.length; i++) {
      prices.push(price_objects[i].children[0].data);
    }
    // image links
    for (var i = 0; i &lt; image_objects.length; i++) {
      images.push(image_objects[i].attribs.src.slice(0, -10));
    }
    // name info
    for (var i = 0; i &lt; name_objects.length; i++) {
      names.push(name_objects[i].children[0].data);
    }
    // color info
    for (var i = 0; i &lt; color_objects.length; i++) {
      colors.push(color_objects[i].attribs.src);
    }
  </code></pre>
  
  <p>Lastly, based on the assumption that price, title, image and colors will match up create a product object:</p>
  
  <pre><code>for (var i = 0; i &lt; images.length; i++) {
    items.push({
      id: i,
      name: names[i],
      price: prices[i],
      image: images[i],
      colors: colors[i]
    });
  }
  </code></pre>
  
  <p>This method is slow, error-prone, and very anti-DRY. I was thinking it would be nice if we could grab <code>$('.cat-thu-product')</code> and using a single for-loop extract relevant information from a single product a time. </p>
  
  <p>But have you ever tried traversing the DOM in <strong>jsdom</strong> or <strong>cheerio</strong>? I am not sure how anyone can even comprehend it. Could someone show how would I use this proposed method of scraping, by grabbing <code>$('.cat-thu-product')</code> div element containing all relevant information and then extract necessary data?</p>
  
  <p>Or perhaps there is a better way to do this?</p>