<story><title>200 TB of 1000 genomes data on Amazon S3 as a public dataset</title><url>http://aws.typepad.com/aws/2012/03/the-1000-genomes-project.html</url></story><parent_chain><item><author>breck</author><text>I know very little about genomics and biology and would love it if someone who knows more could answer my silly questions to help me understand this more:&lt;p&gt;- This has genome data from 1,700 people. In theory could a company build a machine that read one of these &quot;files&quot; as input and made a baby as output?&lt;p&gt;- It says the dataset is 200TB. If 1 person has 3 billion dna bases(which I understand can be 1 of 4 values or 2 bits), it seems that even uncompressed the dataset would be 6billion bits (&amp;#60;1GB per person) and so only ~ 1.7TB uncompressed for 1,700 people. What am I missing?</text></item></parent_chain><comment><author>tomkinstinch</author><text>I haven&apos;t looked at the data yet, but I suspect the large data size is due to full sequencer output. Next-gen sequencing methods produce a LOT of noisy overlapping data that can later be reduced to high quality contiguous sequences.&lt;p&gt;When compressed relative to a reference sequence, a ~3.3Gbp human genome can be reduced to ~3.3MiB of data[1].&lt;p&gt;For more information about the methods currently being used, check out these videos from the WUSTL Genome Center:
&lt;a href=&quot;http://gep.wustl.edu/curriculum/course_materials_WU/introduction_to_genomics/nextgen_video_tour&quot; rel=&quot;nofollow&quot;&gt;http://gep.wustl.edu/curriculum/course_materials_WU/introduc...&lt;/a&gt;&lt;p&gt;1. &lt;a href=&quot;http://dhruvbird.com/genome_compression.pdf&quot; rel=&quot;nofollow&quot;&gt;http://dhruvbird.com/genome_compression.pdf&lt;/a&gt;</text></comment>
<story><title>200 TB of 1000 genomes data on Amazon S3 as a public dataset</title><url>http://aws.typepad.com/aws/2012/03/the-1000-genomes-project.html</url></story><parent_chain><item><author>breck</author><text>I know very little about genomics and biology and would love it if someone who knows more could answer my silly questions to help me understand this more:&lt;p&gt;- This has genome data from 1,700 people. In theory could a company build a machine that read one of these &quot;files&quot; as input and made a baby as output?&lt;p&gt;- It says the dataset is 200TB. If 1 person has 3 billion dna bases(which I understand can be 1 of 4 values or 2 bits), it seems that even uncompressed the dataset would be 6billion bits (&amp;#60;1GB per person) and so only ~ 1.7TB uncompressed for 1,700 people. What am I missing?</text></item></parent_chain><comment><author>skosuri</author><text>1000 genomes includes many datasets including sequencing.  That said, I&apos;m sure most of this is sequencing data.  The way sequencing is done is using billions of reads (usually b/w 50-100 nucleotides long), that are mapped back to the genome.  Average coverage of any individual nucleotide is usually 30-50x.  What people usually shared are compressed versions of the mapped reads that show all the sequencing information.  The reason why this is done is that due to the  limitations of the sequencing technologies, how you call particular mutations, structural variations, and deletions depends on the algorithms you use to reconstruct the genome.  Thus the raw data could be in the form of compressed reads, or the compressed mapped reads (or in this case, likely both).  Even this though is a vast reduction in size for the raw data, which are usually images that are used to call each base, which are usually 1000x larger than this data set; these usually aren&apos;t stored now for obvious space constraints.</text></comment>