<story><title>DALL-E, the Metaverse, and Zero Marginal Content</title><url>https://stratechery.com/2022/dall-e-the-metaverse-and-zero-marginal-content/</url></story><parent_chain></parent_chain><comment><author>bduerst</author><text>Zero marginal content already exists with procedural generation (PG).&lt;p&gt;The problem with PG is that it&amp;#x27;s all the same after a while - i.e. one of the chief complaints of &lt;i&gt;No Man&amp;#x27;s Sky&lt;/i&gt; is that all the PG planets look the same after a short period of time.&lt;p&gt;The real value isn&amp;#x27;t zero-marginal content, but zero-marginal narration (or story telling) that breaks new ground.  Piecing together PG or zero-marginal content isn&amp;#x27;t enough, any next step breakthroughs will come with higher-level orchestration of said content.</text></comment>
<story><title>DALL-E, the Metaverse, and Zero Marginal Content</title><url>https://stratechery.com/2022/dall-e-the-metaverse-and-zero-marginal-content/</url></story><parent_chain></parent_chain><comment><author>axg11</author><text>We are still extremely early in this space. This is the mainframe era of generative machine learning.&lt;p&gt;I&amp;#x27;m expecting three big leaps in the next 10 years (in order):&lt;p&gt;1) Generative algorithms reach a level where the content is indistinguishable from human-generated content. Compute: performed on mega-clusters such as used by DALL-E, GPT-3, PaLM.&lt;p&gt;2) DALL-E&amp;#x2F;GPT-3&amp;#x2F;PaLM-level generative algorithms are able to run on personal hardware (phone, laptop)&lt;p&gt;3) Generative algorithms are able to fine-tune&amp;#x2F;train on personal hardware&lt;p&gt;Right now the algorithms are moving much faster than the hardware, hence why we are seeing large language models and gigantic generative models such as DALL-E 2. In time the hardware will catch up. For the next few years, applications of these models will be restricted by the fact that they&amp;#x27;re only accessible through API calls to mega-clusters run by Google, OpenAI, etc.&lt;p&gt;In time the hardware will improve and architectures will become more compute efficient to the point where we can achieve &amp;quot;human-level&amp;quot; (loosely defined) on personal hardware. Real-time generative content will change the way we consume content entirely, especially in the context of AR. Augmenting our view of the physical world with an infinite number of different filters will create infinite use cases for AR.&lt;p&gt;On the large language model side, once LLMs exceed our ability to comprehend libraries of documents, that will change the way we work, the way we perform science and many other things. Imagine querying your team&amp;#x27;s entire library of documents, design reviews and code with prompts such as: &amp;quot;Why was the load balancer for X service designed in this way?&amp;quot;.</text></comment>