<http://schema.org/articleBody>
"Last night, the Internet Archive threw a party; hundreds of Internet Archive supporters, volunteers, and staff celebrated that the site had passed the 10,000,000,000,000,000 byte mark for archiving the Internet. As the non-profit digital library, known for its Wayback Machine service, points out, the organization has thus now saved 10 petabytes of cultural material. \r\n\r\nPut another way, that's 10 million gigabytes or 10 thousand terabytes. You might be wondering why this news only came out today; there are two reasons: first off, Archive.org posted about it today. Secondly, and this may be the real reason everything was delayed, is that Archive.org was down for a good chunk yesterday:\r\n\r\n\r\n\r\nWhat happened? This statement provides a hint:\r\n\r\n\r\nThe only thing missing was electricity; the building lost all power just as the presentation was to begin. Thanks to the creativity of the Archive\u2019s engineers and a couple of ridiculously long extension cords that reached a nearby house, the show went on.\r\n\r\n\r\nAnyway, in light of the milestone achieved yesterday and announced today, the organization also announced two more achievements: the first complete literature of a people went online (Balinese) and a &amp;quot;full 80 Terabyte web crawl to researchers.&amp;quot;\r\n\r\nThe Internet Archive says it is doing so because it wants to see how others might learn from all the content it accumulates. The crawl consists of WARC files containing captures of about 2.7 billion URIs from last year. The files contain text content and any media that the organization captured, including images, flash, videos, and so on.\r\n\r\n\r\n\r\nHere are the technical details:\r\n\r\n\r\nCrawl start date: 09 March, 2011\r\nCrawl end date: 23 December, 2011\r\nNumber of captures: 2,713,676,341\r\nNumber of unique URLs: 2,273,840,159\r\nNumber of hosts: 29,032,069\r\n\r\n\r\nThe seed list for this particular crawl was a list of Alexa's top 1 million Web sites, retrieved close to the crawl start date. The scope of the crawl was only limited by a few manually excluded sites. \r\n\r\nSome points to note:\r\n\r\nHowever this was a somewhat experimental crawl for us, as we were using newly minted software to feed URLs to the crawlers, and we know there were some operational issues with it.  For example, in many cases we may not have crawled all of the embedded and linked objects in a page since the URLs for these resources were added into queues that quickly grew bigger than the intended size of the crawl (and therefore we never got to them).  We also included repeated crawls of some Argentinian government sites, so looking at results by country will be somewhat skewed.\r\n\r\n\r\nIf you want to get access to the crawl data, contact info[at]archive[dot]org. The organization isn't making any promises, but it does say it will consider everyone who explains why they want to use this particular archive.\r\n\r\nImage credit: Jenny Sliwinski"^^<http://www.w3.org/2001/XMLSchema#string>
<http://thenextweb.com/insider/2012/10/26/the-internet-archive-has-saved-over-10000000000000000-bytes-of-the-web/>