15. A computer-implemented method for crawling structured application data of an application by a search engine, the method comprising: generating, with one or more processors associated with one or more computer systems hosting a search engine, a first request for application data as a seed link sourcing a business object associated with the application; sending, from the one or more computer systems hosting the search engine, the first request for the sourced business object associated with the application; receiving, at the one or more computer systems, a control feed generated in response to the seed link and a crawlable definition for the business object associated with the application, the crawlable definition including information specifying a query selecting one or more attributes of the business object associated with the application, the control feed comprising information dividing a data feed associated with a crawlable document into a set of transferable batches according to predetermined criteria and one or more re-entry links for each transferable batches in the set of transferable batches; crawling, with the one or more processors associated with the one or more computer systems, the control feed to push the one or more re-entry links onto a job queue maintained by the one or more computer systems; generating, with the one or more processors associated with the one or more computer systems, a second request for structured application data as a re-entry link associated with the feed document, the second request being for a re-entry link of the one or more re-entry links in the job queue; sending, from the one or more computer system, the second request for structured application data; receiving, at the one or more computer systems, a crawlable batch from the data feed where the batch is requested in the re-entry link as a batch link; receiving, at the one or more computer systems, a crawlable related document where the crawlable related document is requested in the re-entry link as a related document link; and receiving, at the one or more computer systems, a crawlable dependent document where the crawlable related document is requested in the re-entry link as a dependent document link.
15. A computer-implemented method for crawling structured application data of an application by a search engine, the method comprising: generating, with one or more processors associated with one or more computer systems hosting a search engine, a first request for application data as a seed link sourcing a business object associated with the application; sending, from the one or more computer systems hosting the search engine, the first request for the sourced business object associated with the application; receiving, at the one or more computer systems, a control feed generated in response to the seed link and a crawlable definition for the business object associated with the application, the crawlable definition including information specifying a query selecting one or more attributes of the business object associated with the application, the control feed comprising information dividing a data feed associated with a crawlable document into a set of transferable batches according to predetermined criteria and one or more re-entry links for each transferable batches in the set of transferable batches; crawling, with the one or more processors associated with the one or more computer systems, the control feed to push the one or more re-entry links onto a job queue maintained by the one or more computer systems; generating, with the one or more processors associated with the one or more computer systems, a second request for structured application data as a re-entry link associated with the feed document, the second request being for a re-entry link of the one or more re-entry links in the job queue; sending, from the one or more computer system, the second request for structured application data; receiving, at the one or more computer systems, a crawlable batch from the data feed where the batch is requested in the re-entry link as a batch link; receiving, at the one or more computer systems, a crawlable related document where the crawlable related document is requested in the re-entry link as a related document link; and receiving, at the one or more computer systems, a crawlable dependent document where the crawlable related document is requested in the re-entry link as a dependent document link. 18. The method of claim 15 , further comprising: indexing, with the one or more computer systems, a content of the received crawlable dependent document according to an indexing rule set out in metadata of the received dependent document.