<story><title>&quot;Hallucinating&quot; AIs sound creative, but let&apos;s not celebrate being wrong</title><url>https://thereader.mitpress.mit.edu/hallucinating-ais-sound-creative-but-lets-not-celebrate-being-wrong/</url></story><parent_chain><item><author>jshen</author><text>&amp;gt; &amp;quot;The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time.&amp;quot;&lt;p&gt;I always find this point a bit odd because humans aren&amp;#x27;t &amp;quot;optimized&amp;quot; to be correct either.&lt;p&gt;&amp;gt; &amp;quot;LLMs have no understanding of the underlying reality&amp;quot;&lt;p&gt;I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if could only answer with values from one column and it did exactly as I asked. It&amp;#x27;s hard for me to see things like that and not believe it has an understanding at some level.</text></item><item><author>happytiger</author><text>The LLM AI technology generation is &lt;i&gt;optimized to be fluently conversational&lt;/i&gt; and not to be &lt;i&gt;factually correct&lt;/i&gt; all the time.&lt;p&gt;1) Hallucinations often appear because LLMs are designed to &lt;i&gt;create fluent, coherent text.&lt;/i&gt;&lt;p&gt;2) LLMs have no understanding of the underlying reality that language describes.&lt;p&gt;3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt.&lt;p&gt;It sacrifices accuracy for being good at conversations &lt;i&gt;as it is designed to do.&lt;/i&gt; All these criticisms of hallucinations are missing the  point.&lt;p&gt;Generative AI is &lt;i&gt;generative&lt;/i&gt; and basically is a specialist at making things up. It’s going to take things like multiprompting, network AIs that fact check output, and a host of other technologies or even entirely new models of AI to solve these problems, but don’t make the mistake of thinking that the system is &lt;i&gt;supposed to be working without hallucinations&lt;/i&gt; right now — that’s not what it’s optimized for.</text></item></parent_chain><comment><author>pixl97</author><text>&amp;gt;I always find this point a bit odd because humans aren&amp;#x27;t &amp;quot;optimized&amp;quot; to be correct either.&lt;p&gt;It depends which subsystem.&lt;p&gt;The older the system in the body the less likely it is to have a high error rate, otherwise we&amp;#x27;d die from cancer at a much higher rate or injure ourselves far more often. Of course this also depends on the definition of &amp;#x27;correct&amp;#x27;, if the system never changed we&amp;#x27;d never evolve.&lt;p&gt;&amp;gt;&amp;gt; &amp;quot;LLMs have no understanding of the underlying reality&amp;quot;&lt;p&gt;This statement has always bothered me because it really depends on what you mean by &amp;#x27;underlying reality&amp;#x27;. How many layers are we talking about? What does understanding mean? Because at the end of the day, humans don&amp;#x27;t really understand &amp;#x27;underlying reality, we just have a particular set of input devices we take in information and do some transformations on it... Where is the understanding happening?</text></comment>
<story><title>&quot;Hallucinating&quot; AIs sound creative, but let&apos;s not celebrate being wrong</title><url>https://thereader.mitpress.mit.edu/hallucinating-ais-sound-creative-but-lets-not-celebrate-being-wrong/</url></story><parent_chain><item><author>jshen</author><text>&amp;gt; &amp;quot;The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time.&amp;quot;&lt;p&gt;I always find this point a bit odd because humans aren&amp;#x27;t &amp;quot;optimized&amp;quot; to be correct either.&lt;p&gt;&amp;gt; &amp;quot;LLMs have no understanding of the underlying reality&amp;quot;&lt;p&gt;I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if could only answer with values from one column and it did exactly as I asked. It&amp;#x27;s hard for me to see things like that and not believe it has an understanding at some level.</text></item><item><author>happytiger</author><text>The LLM AI technology generation is &lt;i&gt;optimized to be fluently conversational&lt;/i&gt; and not to be &lt;i&gt;factually correct&lt;/i&gt; all the time.&lt;p&gt;1) Hallucinations often appear because LLMs are designed to &lt;i&gt;create fluent, coherent text.&lt;/i&gt;&lt;p&gt;2) LLMs have no understanding of the underlying reality that language describes.&lt;p&gt;3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt.&lt;p&gt;It sacrifices accuracy for being good at conversations &lt;i&gt;as it is designed to do.&lt;/i&gt; All these criticisms of hallucinations are missing the  point.&lt;p&gt;Generative AI is &lt;i&gt;generative&lt;/i&gt; and basically is a specialist at making things up. It’s going to take things like multiprompting, network AIs that fact check output, and a host of other technologies or even entirely new models of AI to solve these problems, but don’t make the mistake of thinking that the system is &lt;i&gt;supposed to be working without hallucinations&lt;/i&gt; right now — that’s not what it’s optimized for.</text></item></parent_chain><comment><author>pdonis</author><text>&lt;i&gt;&amp;gt; humans aren&amp;#x27;t &amp;quot;optimized&amp;quot; to be correct either&lt;/i&gt;&lt;p&gt;Humans at least have a &lt;i&gt;concept&lt;/i&gt; of &amp;quot;being correct&amp;quot;, even if we don&amp;#x27;t always set that as our primary goal when communicating.&lt;p&gt;LLMs don&amp;#x27;t even have a &lt;i&gt;concept&lt;/i&gt; of &amp;quot;being correct&amp;quot;. That would require having a concept of an &amp;quot;external world&amp;quot; that text refers to, which LLMs don&amp;#x27;t have. &lt;i&gt;All&lt;/i&gt; they have is the text in their training data.</text></comment>