title: Web Cralwer Algorithm: depth?
body: <p>I'm working on a crawler and need to understand exactly what is meant by "link depth". Take nutch for example: <a href="http://wiki.apache.org/nutch/NutchTutorial" rel="nofollow">http://wiki.apache.org/nutch/NutchTutorial</a></p>
  
  <blockquote>
    <p>depth indicates the link depth from the root page that should be
    crawled.</p>
  </blockquote>
  
  <p>So, say I have the domain <code>www.domain.com</code> and wanted to crawl a depth of, say, <code>3</code> -- what do I need to do? If a site could be represented as a binary tree, then it wouldn't be a problem I think.</p>