title: Get some or all (sub)page URLs of a website by a particular pattern
body: <p>Suppose we have a website called <code>http://www.example.com</code>. I would like to get a list of its URI pages (just the URLs themselves, not URLs inside those URLs) - either all of them (including all subdomains and all subpages), or just some of them provided that they follow a particular globbing and/or regex pattern.</p>
  
  <p>So, for example, I'm looking for something that gets all URLs (just the URL addresses themselves) that follow a pattern such as <code>http://*.example.com/*</code>. I'm aware that globbing in Linux (e.g. via the shell) is (mostly or fully?) limited to local files and directories (correct me if I'm wrong).</p>
  
  <p>How can I achieve this?</p>
  
  <p>I suppose that something related (although not quite the same?) is discussed here: <a href="https://stackoverflow.com/questions/1439326/how-to-find-all-links-pages-on-a-website">How to find all links / pages on a website</a>.</p>
  
  <p><strong>P.S.</strong> All of the URLs are part of a website that is made of static webpages only. I'm not sure if it's even possible to do the same thing with websites that are made of dynamic webpages... Also, I'm not sure if any URLs with query strings in them (e.g. <code>http://www.example.com/?=abc&amp;xyz</code>) can be captured at all in such a way.</p>