What role should text-generating large language models (LLMs) have in the scientific research process? According to a team of Oxford scientists, the answer — at least for now — is: pretty much none.
In a new essay, researchers from the Oxford Internet Institute argue that scientists should abstain from using LLM-powered tools like chatbots to assist in scientific research on the grounds that AI's penchant for hallucinating and fabricating facts, combined with the human tendency to anthropomorphize the human-mimicking word engines, could lead to larger information breakdowns — a fate that could ultimately threaten the fabric of science itself.
"Our tendency to anthropomorphize machines and trust models as human-like truth-tellers, consuming and spreading the bad information that they produce in the process," the researchers write in the essay, which was published this week in the journal Nature Human Behavior, "is uniquely worrying for the future of science."
The scientists' argument hinges on the reality that LLMs and the many bots that the technology powers aren't primarily designed to be truthful. As they write in the essay, sounding truthful is but "one element by which the usefulness of these systems is measured." Characteristics including "helpfulness, harmlessness, technical efficiency, profitability, [and] customer adoption" matter, too.
"LLMs are designed to produce helpful and convincing responses," they continue, "without any overriding guarantees regarding their accuracy or alignment with fact."
Put simply, if a large language model — which, above all else, is taught to be convincing — comes up with an answer that's persuasive but not necessarily factual, the fact that the output is persuasive will override its inaccuracy. In an AI's proverbial brain, simply saying "I don't know" is less helpful than providing an incorrect response.
But as the Oxford researchers lay out, AI's hallucination problem is only half the problem. The Eliza Effect, or the human tendency to read way too far into human-sounding AI outputs due to our deeply mortal proclivity to anthropomorphize everything around us, is a well-documented phenomenon. Because of this effect, we're already primed to put a little too much trust in AI; couple that with the confident tone these chatbots so often take, and you have a perfect recipe for misinformation. After all, when a human gives us a perfectly bottled, expert-sounding paraphrasing in response to a query, we're probably less inclined to use the same critical thinking in our fact-checking as we might when we're doing our own research.
Importantly, the scientists do note "zero-shot translation" as a scenario in which AI outputs might be a bit more reliable. This, as Oxford professor and AI ethicist Brent Mittelstadt told EuroNews, refers to when a model is given "a set of inputs that contain some reliable information or data, plus some request to do something with that data."
"It's called zero-shot translation because the model has not been trained specifically to deal with that type of prompt," Mittelstadt added. So, in other words, a model is more or less rearranging and parsing through a very limited, trustworthy dataset, and not being used as a vast, internet-like knowledge center. But that would certainly limit its use cases, and would demand a more specialized understanding of AI tech — much different from just loading up ChatGPT and firing off some research questions.
And elsewhere, the researchers argue, there's an ideological battle at the core of this automation debate. After all, science is a deeply human pursuit. To outsource too much of the scientific process to automated AI labor, the Oxforders say, could undermine that deep-rooted humanity. And is that something we can really afford to lose?
"Do we actually want to reduce opportunities for writing, thinking critically, creating new ideas and hypotheses, grappling with the intricacies of theory and combining knowledge in creative and unprecedented ways?" the researchers write. "These are the inherently valuable hallmarks of curiosity-driven science."
"They are not something that should be cheaply delegated to incredibly impressive machines," they continue, "that remain incapable of distinguishing fact from fiction."
Share This Article
Ni nini jukumu maandishi-kuzalisha kubwa lugha mifano (LLMs) kuwa katika mchakato wa utafiti wa kisayansi? Kulingana na kikundi cha wanasayansi wa Oxford, jibu - angalau kwa sasa - ni: hakuna hata moja. Katika makala yao, watafiti kutoka Taasisi ya Mtandao ya Oxford wanasema wanasayansi wanapaswa kujiepusha na kutumia zana za LLM kama vile chatbots kusaidia katika utafiti wa kisayansi kwa sababu ya upendeleo wa AI kwa kugundua na kutengeneza ukweli, pamoja na mwelekeo wa binadamu wa anthropomorphize injini za maneno za binadamu, inaweza kusababisha uharibifu mkubwa wa habari - hatima ambayo inaweza hatimaye kutishia muundo wa sayansi yenyewe. "Uwezo wetu wa kufananisha mashine na kuamini mifano kama watu wanaosema ukweli, kutumia na kueneza habari mbaya wanayozalisha katika mchakato huo, ""waandishi wa utafiti waliandika katika insha iliyochapishwa wiki hii katika jarida la Nature Human Behavior." "Hii ni kwa sababu ya ukweli kwamba ""LMS"" na ""bots"" nyingi ambazo nguvu za teknolojia hazijaundwa hasa kuwa za kweli." Kama wanavyosema katika insha hiyo, kuonekana kuwa wa kweli ni "kiwango kimoja tu cha kupima manufaa ya mifumo hiyo". Sifa ikiwa ni pamoja na "usaidizi, harmlessness, ufanisi wa kiufundi, faida, [na] kupitishwa kwa wateja" ni muhimu, pia. "LLMs zimetengenezwa kutoa majibu yenye kusaidia na yenye kusadikisha", wanaendelea, "bila dhamana yoyote ya juu kuhusu usahihi wao au usawa na ukweli". Kwa ufupi, ikiwa mfano mkubwa wa lugha - ambayo, juu ya kila kitu kingine, hufundishwa kuwa yenye kusadikisha - inakuja na jibu ambalo ni la kusadikisha lakini sio lazima la ukweli, ukweli kwamba pato ni la kusadikisha litabadilisha ukosefu wake wa usahihi. "Kwa mfano, katika ubongo wa AI, kusema tu ""Sijui"" ni muhimu kuliko kutoa jibu lisilo sahihi." Lakini watafiti wa Oxford wanasema kuwa tatizo la kupofuka kwa akili ni nusu tu ya tatizo. Athari ya Eliza, au mwelekeo wa binadamu wa kusoma mbali sana katika matokeo ya AI ya sauti ya binadamu kutokana na mwelekeo wetu wa kufa sana wa anthropomorphize kila kitu karibu nasi, ni jambo la kumbukumbu. Kwa sababu ya athari hii, tayari tumejitayarisha kuweka imani kidogo sana katika AI; jozi hiyo na sauti ya kujiamini ambayo mazungumzo haya mara nyingi huchukua, na una kichocheo kamili kwa habari potofu. Baada ya yote, wakati binadamu anatoa sisi kikamilifu chupa, mtaalamu-kuonekana paraphrasing katika kujibu swali, sisi ni pengine chini ya mwelekeo wa kutumia mawazo sawa muhimu katika ukweli wetu kuangalia kama tunaweza wakati sisi ni kufanya utafiti wetu wenyewe. "Wanasayansi wanasema ""utafsiri wa risasi sifuri"" ni hali ambayo matokeo ya AI yanaweza kuwa ya kuaminika zaidi." "Profesa wa Oxford na mtaalamu wa maadili ya AI Brent Mittelstadt aliiambia EuroNews, ""Hii inahusu wakati mfano unapewa seti ya pembejeo ambazo zina habari au data ya kuaminika, pamoja na ombi la kufanya kitu na data hiyo.""" "Inaitwa zero-shot tafsiri kwa sababu mfano haijafundishwa hasa kushughulika na aina hiyo ya mwongozo", Mittelstadt aliongeza. Kwa maneno mengine, mfano ni zaidi au chini ya kurekebisha na kuchanganua kupitia data ndogo sana, ya kuaminika, na sio kutumika kama kituo kikubwa cha maarifa kama mtandao. Lakini hiyo bila shaka itapunguza kesi zake za matumizi, na itahitaji uelewa wa maalumu zaidi wa teknolojia ya AI - tofauti sana na kupakia tu ChatGPT na kufyatua maswali ya utafiti. Na mahali pengine, watafiti wanasema, kuna vita vya kiitikadi katika kiini cha mjadala huu wa automatisering. Baada ya yote, sayansi ni kazi ya kibinadamu sana. "Kutoa kazi nyingi za kisayansi kwa kazi ya AI ya kiotomatiki, ""Wanafunzi wa Oxford wanasema, inaweza kudhoofisha ubinadamu huo wenye mizizi ya kina." Na je, hilo ni jambo ambalo kwa kweli twaweza kupoteza? "Je, kweli tunataka kupunguza fursa za kuandika, kufikiri kwa ukali, kuunda mawazo mapya na nadharia, kukabiliana na utata wa nadharia na kuchanganya maarifa kwa njia za ubunifu na zisizo za kawaida?""" watafiti wanaandika. "Hizi ndizo alama zenye thamani za asili za sayansi inayoendeshwa na udadisi". "Hawa si kitu ambacho kinapaswa kupewa mashine zenye kuvutia sana kwa bei nafuu", wanaendelea kusema, "ambazo bado haziwezi kutofautisha ukweli na uwongo". Shiriki Makala Hii