title: Finding all pages on domain with NodeJS
body: <p>I'm trying to find all the pages on a domain with Node.
  I was searching on Stackoverflow, but all i found is this thread for Ruby: <a href="https://stackoverflow.com/questions/10272920/find-all-the-web-pages-in-a-domain-and-its-subdomains">Find all the web pages in a domain and its subdomains</a> - I have the same question, but for Node.
  I've also googled the question, but all I find are scrapers that do not find the links to scrape themselves. I was also searching for something like "sitemap generator", "webpage robot", "automatic scraper", "getting all pages on domain with Node" but it didn't bring any result. </p>
  
  <p>I have a scraper that needs an array of links it will be processing and for example I have a page www.example.com/products/ where I want to find all existing sub-pages, e.g. www.example.com/products/product1.html, www.example.com/products/product2.html etc.</p>
  
  <p>Could you give me a hint how can I implement it in Node?</p>