6. The method as defined in claim 1 , wherein the retrieving the dynamic data document further comprises performing the following sub-steps of: initializing a first list with seed values; checking if there are any URLs to be processed and in response that any URL exists to be processed then performing the following sub-steps of: determining if a URL is in a second list; and in response that a URL is not in the second list then performing the following sub-steps of: inserting the URL into the first list; scheduling the URL for crawling; crawling the URL when scheduled to do so; removing the URL from the first list after the scheduled crawling; entering the URL into the second list; and repeating the checking step until there are no more URLs to be processed; where if the determining step determines that the URL is in the second list then repeating the checking step until there are no more URLs to be processed.
8. The method as defined in claim 6 , wherein the sub-step of determining if a URL is in a second list further includes the second list being a visited pool.