Background information about the Artificial Intelligence survey and initial results
Some of us organized in 2020 a survey of all NIH grantees about the COVID-19 response. We received more than 4,000 replies and Nature wrote a piece about it:
This is an example of how community-wide surveys can have positive effects in science.
The current survey promotes a discussion about the control of Artificial Intelligence (AI) by the scientific community. The discussion focuses on recent AI developments important for science, including Large Language Models (LLMs) like ChatGPT and Bard and scientific tools like AlphaFold and AlphaMissense. We have already obtained opinions from biomedical scientists and from economists and we plan to extend the discussion to specialists in other scientific disciplines and to historians. This is not a problem that a single individual or even a single scientific field can solve in isolation. Most likely, it will require a form of collective thinking in which opinions are first shared and then revised after reading what specialists in other fields have said.
In addition to the survey we also present here research based on published documents, interviews and perspectives from experts.
Can we really expect scientists to act as a community?
There are several reasons why this is a realistic prospect:
- We show historical sources and an analysis describing times when scientist in molecular biology were part of a community freely exchanging ideas. This later changed, in part due to increased competitive concerns. In the case of the topic of this survey, however, there would be no individual advantage in secrecy. In these historical sections we also present descriptions of the beginning of open science at the time of the Scientific Revolution, in the seventeenth century, and of its effect on economic development.
- Another change compared to the early times of molecular biology is the increase in size of the scientific community. There is however an example of a large scientific field, particle physics, that has developed a mechanism for community-wide discussions of ideas, when faced with problems that require concerted efforts.
- Global collaborations in biology are emerging when the problem to be addressed requires it. One of these is the Human Cell Atlas. You can read an interview with Aviv Regev and Sarah Teichmann, the leaders of this initiative, which now involves more than 3,000 scientists from 95 different countries. Another example is the response to the COVID-19 pandemics, described by Alessandro Sette.
- The development of Artificial Intelligence challenges human understanding. Historians and social scientists have found that external threats often make groups more cohesive, strengthening group identity, in this case human identity. We are obtain opinions from historians. An example is the interview with Carlo Ginzburg.
Why would more transparency be helpful?
We cannot always explain the reasons on which AI statements are based. As discussed in the case of medical applications by Ghassemi et al (1) this is in part due to the nature of the algorithms, and in some cases we can mainly expect validation rather than explanation. Obstacles to understanding, however, are also due to incomplete disclosure by the developers of the data used for training, of the details of the methods and of the validation steps. Burnell et al (2) show how published reports of AI systems performance often do not provide sufficient information for a complete evaluation. In a recent article (3) Melanie Mitchell stated that:
"To scientifically evaluate claims of humanlike and even superhuman machine intelligence, we need more transparency on the ways these models are trained, and better experimental methods and benchmarks. Transparency will rely on the development of open-source (rather than closed, commercial) AI models. "
How does interacting with humans contribute to the training of AI systems?
An example is "reinforcement learning from human feedback". This method has been mentioned as one of the main factors for the success of ChatGPT (4). It has been developed by scientists at OpenAI with the aim of training AI systems "to do what a given set of humans want them to do." (5) Similar approaches are likely to be used in more advanced systems (6).
Who should control AI systems?
Different parts of society can play a role in AI control (7). The scientific community can participate in this control and make sure that fundamental scientific knowledge remains a public resource.
The case of drug development shows how private companies can contribute to specific applications but also benefit from openly shared fundamental biological knowledge. AI systems contain scientfic knowledge in a form that is not equivalent to that available from articles, books or individual human experts.
Survey results (July-August 2023) - biomedical scientists
The survey of biomedical scientists received 187 responses. 80% of respondents had previously received an NIH grant (among them a Nobel Prize winner). Most of the other respondents were international scientists that have published papers included in Pubmed.
After a short introduction the survey asked 3 questions and this page was mentioned for those that wanted more background information. Here are the questions and a summary of the responses:
Artificial Intelligence (AI) systems contain a large and increasing amount of fundamental knowledge about human biology. Interacting with AI systems contributes to training them; an example of this is reinforcement learning from human feedback.
Q1. Do you think that it is useful for the biomedical community to discuss the control of AI systems?
96% of respondents answered YES to this fixed choice question.
Q2. Which are the advantages and disadvantages of promoting the development of AI systems that are more transparent, and where the biomedical community participates in the control?
You can see here all the individual responses to this question.
Scientists provided a wider range of ideas compared to the responses from AI systems mentioned above.
Q3. Should biomedical scientists interact preferentially with AI systems that have these features?
You can see here all the individual replies to this question.
55% of respondents answered YES or gave an equivalent response, 4% answered NO, 29% made a more nuanced comment and 12% did not answer this question.
You can also read a summary of the replies given by biomedical scientists to both free text questions.
Several Editors of scientific journals have shared with us their perspectives about the recent developments of AI in science.
We have also received perspectives from computational biologists and bioengineers.
Demis Hassabis is the CEO of Google DeepMind and he has won the 2023 Lasker Award in Basic Medical Science for his contribution to AlphaFold, an AI system that predicts the three-dimensional structure of proteins from the amino acid sequence. His vision is that large language models (LLMs) could use language to call one or more AI tools, like AlphaFold, after receiving a request from a user. Another AI tool developed by his group, published in Science in September 2023, is AlphaMissense, which predicts the pathogenicity of all possible human single amino acid substitutions. All the components of the AlphaFold AI model were shared openly, but in the case of AlphaMissense the trained weights, a set of parameters essential for running the model, were not shared. This decision and similar ones adopted for several LLMs are example of the limitations to open science that are emerging in the AI field, even when AI is used for fundamental science.
Survey results (October 2023) - economists
The survey of economists received 33 replies. 62% of the respondents are affiliated with NBER (National Bureau of Economic Research), one of the foremost economic think tanks. These are primarily well-known academics from North American Universities. We also received replies from an international group of economists affiliated with Universities in US, Canada, UK, Norway, Italy, Denmark, India, Japan, New Zealand and with the World Bank and the IMF.
We asked the same questions that had been answered by biomedical scientists, now directed to scientists in general, and a few additional questions for economists:
Artificial Intelligence (AI) systems contain a large and increasing amount of fundamental knowledge about science. Interacting with AI systems contributes to training them; an example of this is reinforcement learning from human feedback.
General questions for all scientists:
Q1. Do you think that it is useful for the scientific community to discuss the control of AI systems?
Q2. Which are the advantages and disadvantages of promoting the development of AI systems that are more transparent, and where the scientific community participates in the control?
Q3. Should scientists interact preferentially with AI systems that have these features (more transparency and control participation)?
Additional introduction and questions for economists:
Since the beginning of the Scientific Revolution in the seventeenth century, fundamental scientific knowledge of Nature has been a public resource, while applied knowledge of commercial value has either been protected by patents, which require disclosure in exchange for temporary exclusivity, or kept as a secret. Several large AI systems important for both fundamental and applied science, including Large Language Models (LLMs) and other AI tools, are now being developed by a few companies that keep key quantitative parameters as a proprietary secret. These AI systems require large investments.
Q4. How might the rise of these private AI systems affect the flow of ideas? What does this imply about data and scientific integrity in the future? What does this imply about the pace and distribution of economic growth?
Q5. What role should the public sector play in regulating these AI models, to maximize their benefits and minimize their risks?
Q6. What would be the economic consequences if public AI systems, dedicated to fundamental scientific knowledge, would co-exist with private AI efforts, focused on specific applications?
All economist responders answered YES to Question 1. You can read a summary of the responses and all the individual replies. We used AI (ChatGPT 3.5) to assist in the survey summaries, with final human editing.
1- Ghassemi M, Oakden-Rayner L, Beam AL. "The false hope of current approaches to explainable artificial intelligence in health care." The Lancet Digital Health. 2021 Nov 1;3(11):e745-50.
2- Burnell R, Schellaert W, Burden J, Ullman TD, Martinez-Plumed F, Tenenbaum JB, Rutar D, Cheke LG, Sohl-Dickstein J, Mitchell M, Kiela D. "Rethink reporting of evaluation results in AI." Science. 2023 Apr 14;380(6641):136-8.
3- Mitchell M. "How do we know how smart AI systems are?". Science. 2023 Jul 13;381(6654):adj5957.
4- Heaven WD "The inside story of how ChatGPT was built from the people who made it."
MIT Technology Review 2023
5- Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A, Schulman J. "Training language models to follow instructions with human feedback." Advances in Neural Information Processing Systems. 2022 Dec 6;35:27730-44.
6- Thirunavukarasu AJ, Ting DS, Elangovan K, Gutierrez L, Tan TF, Ting DS. "Large language models in medicine." Nature Medicine. 2023 Jul 17:1-1
7- Taddeo M, Floridi L. "How AI can be a force for good." Science. 2018 Aug 24;361(6404):751-2.
"Kulingana na utafiti wa ""Artificial Intelligence"" na ""Utafiti wa Ujasusi wa Artificial"" uliofanywa na NIH, utafiti huo ulifanyika mnamo 2020 na ulijumuisha washiriki wa NIH wote waliopokea ruzuku ya COVID-19." "Tumepokea majibu zaidi ya 4,000 na Nature imeandika kipande kuhusu hilo: ""Hii ni mfano wa jinsi uchunguzi wa jamii nzima unaweza kuwa na athari nzuri katika sayansi." Utafiti wa sasa unakuza majadiliano juu ya udhibiti wa Akili bandia (AI) na jamii ya kisayansi. Mafunzo haya yanazungumza juu ya maendeleo ya hivi karibuni ya AI muhimu kwa sayansi, pamoja na Mifano Kubwa ya Lugha (LLMs) kama ChatGPT na Bard na zana za kisayansi kama AlphaFold na AlphaMissSense. """Tumepata maoni kutoka kwa wanasayansi wa biomedical na kutoka kwa wachumi na tunapanga kupanua majadiliano kwa wataalamu wa taaluma zingine za kisayansi na wanahistoria." Hii si tatizo ambalo mtu mmoja au hata uwanja mmoja wa kisayansi unaweza kutatua kwa kujitenga. Inawezekana zaidi kwamba itahitaji aina fulani ya kufikiri kwa pamoja, ambapo maoni yanashirikiwa kwanza na kisha kurekebishwa baada ya kusoma yale ambayo wataalamu katika nyanja nyingine wamesema. Mbali na uchunguzi sisi pia kuwasilisha hapa utafiti kulingana na hati zilizochapishwa, mahojiano na mtazamo kutoka kwa wataalamu. Je, kwa kweli twaweza kutarajia wanasayansi watende kama jumuiya? Kuna sababu kadhaa kwa nini hii ni matarajio ya kweli: Tunaonyesha vyanzo vya kihistoria na uchambuzi unaoelezea nyakati wakati mwanasayansi wa biolojia ya molekuli alikuwa sehemu ya jamii ya kubadilishana mawazo kwa uhuru. Hii baadaye ilibadilika, kwa sehemu kutokana na wasiwasi wa ushindani ulioongezeka. Hata hivyo, katika kisa cha mada ya uchunguzi huu, hakungekuwa na faida ya mtu binafsi katika siri. Katika sehemu hizi za kihistoria, tunaelezea pia mwanzo wa sayansi ya wazi wakati wa Mapinduzi ya Sayansi katika karne ya kumi na saba na athari zake kwa maendeleo ya kiuchumi. - Mabadiliko mengine ikilinganishwa na nyakati za mapema za biolojia ya molekuli ni ongezeko la ukubwa wa jamii ya kisayansi. Hata hivyo, kuna mfano wa uwanja mkubwa wa kisayansi, fizikia ya chembe, ambayo imebuni utaratibu wa majadiliano ya jamii nzima ya mawazo, wakati inakabiliwa na matatizo ambayo yanahitaji juhudi za pamoja. - Ushirikiano wa kimataifa katika biolojia ni kuibuka wakati tatizo kushughulikiwa inahitaji hivyo. Mojawapo ni Atlas ya Chembe za Binadamu. "Kama ilivyoelezwa hapo juu, utafiti huo ulifanywa na Aviva Regev na Sarah Teichmann, ambao ni viongozi wa ""Mfumo wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti wa Utafiti.""" Mfano mwingine ni majibu ya janga la COVID-19, iliyoelezwa na Alessandro Sette. - Maendeleo ya Artificial Intelligence changamoto ya uelewa wa binadamu. Wanahistoria na wanasayansi wa kijamii wamegundua kwamba vitisho vya nje mara nyingi hufanya makundi yawe na mshikamano zaidi, kuimarisha utambulisho wa kikundi, katika kesi hii utambulisho wa kibinadamu. Tunapata maoni kutoka kwa wanahistoria. Mfano ni mahojiano na Carlo Ginzburg. Kwa nini uwazi zaidi ungekuwa na manufaa? Hatuwezi daima kueleza sababu ambazo taarifa AI ni msingi. Kama ilivyojadiliwa katika kesi ya maombi ya matibabu na Ghassemi et al, hii ni sehemu kutokana na asili ya algorithms, na katika baadhi ya kesi tunaweza hasa kutarajia uthibitisho badala ya maelezo. Hata hivyo, vikwazo kwa uelewa pia ni kutokana na ufunuo kamili na watengenezaji wa data kutumika kwa ajili ya mafunzo, maelezo ya mbinu na hatua ya uthibitisho. Burnell et al. (2) kuonyesha jinsi ripoti kuchapishwa ya utendaji AI mifumo mara nyingi kutoa habari ya kutosha kwa ajili ya tathmini kamili. "Mwanahistoria wa Marekani Melanie Mitchell alisema: ""Kwa ajili ya kutathmini kisayansi madai ya ubinadamu na hata ubinadamu wa juu wa akili ya mashine, tunahitaji uwazi zaidi juu ya njia hizi mifano ni mafunzo, na mbinu bora ya majaribio na benchmarks." Uwazi utategemea maendeleo ya chanzo wazi (mbele ya kufungwa, kibiashara) AI mifano. " Jinsi gani kuingiliana na wanadamu huchangia mafunzo ya mifumo ya AI? Mfano ni "kujifunza kwa kuimarisha kutokana na maoni ya kibinadamu". Njia hii imetajwa kama moja ya sababu kuu kwa ajili ya mafanikio ya ChatGPT (4). "Kama ilivyoelezwa hapo juu, OpenAI ni mfumo wa AI ambao hutengenezwa na ""kila mtu anayetaka kufanya.""" (5) Mbinu kama hizo huenda zikatumiwa katika mifumo ya hali ya juu zaidi (6). Ni nani anayepaswa kudhibiti mifumo ya AI? Sehemu tofauti za jamii zinaweza kucheza jukumu katika udhibiti wa AI (7). Jumuiya ya kisayansi inaweza kushiriki katika udhibiti huu na kuhakikisha kwamba maarifa ya msingi ya kisayansi hubaki rasilimali ya umma. Mfano wa maendeleo ya dawa inaonyesha jinsi makampuni binafsi inaweza kuchangia maombi maalum lakini pia kufaidika na wazi pamoja maarifa ya msingi ya kibiolojia. Mifumo ya AI ina maarifa ya kisayansi katika fomu ambayo haifai kwa ile inayopatikana kutoka kwa makala, vitabu au wataalamu wa binadamu. Uchunguzi wa utafiti wa Biomedical Science (Juni-Agosti 2023) ulipata majibu ya 187 80% ya washiriki walikuwa wamepokea ruzuku ya NIH (kati yao mshindi wa Tuzo ya Nobel). Wengi wa waliohojiwa wengine walikuwa wanasayansi wa kimataifa ambao wamechapisha karatasi zilizojumuishwa katika Pubmed. Baada ya kuanzishwa kwa muda mfupi, uchunguzi ulijibu maswali matatu na ukurasa huu ulitajwa kwa wale ambao walitaka habari zaidi ya msingi. "Hii ni orodha ya maswali na majibu: ""Mifumo ya akili bandia (AI) ina kiasi kikubwa na kuongezeka kwa maarifa ya msingi kuhusu biolojia ya binadamu." Kuingiliana na mifumo ya AI inachangia mafunzo yao; mfano wa hii ni kujifunza kuimarisha kutoka kwa maoni ya binadamu. Q1 - Uwezo wa kuamua Je, unafikiri ni muhimu kwa jamii ya biomedical kujadili udhibiti wa mifumo ya AI? 96% ya watu waliojibu swali la swali la chaguo la msingi walijibu NDIYO. Q2 - Q2 Ni nini faida na hasara za kukuza maendeleo ya mifumo ya AI ambayo ni wazi zaidi, na ambapo jamii ya biomedical inashiriki katika udhibiti? Unaweza kuona hapa majibu yote ya mtu binafsi kwa swali hili. Wanasayansi walitoa maoni mbalimbali ikilinganishwa na majibu kutoka kwa mifumo ya AI iliyotajwa hapo juu. Q3 - Uwezo wa kuamua Je, wanasayansi wa biomedical wanapaswa kuingiliana kwa upendeleo na mifumo ya AI ambayo ina sifa hizi? Unaweza kuona hapa majibu yote ya mtu binafsi kwa swali hili. 55% ya waliohojiwa walijibu NDIYO au walipa jibu sawa, 4% walijibu NO, 29% walifanya maoni ya kina zaidi na 12% hawakujibu swali hili. Unaweza pia kusoma muhtasari wa majibu yaliyotolewa na wanasayansi wa biomedical kwa maswali yote mawili ya maandishi huru. Mhariri wa jarida la kisayansi ameshiriki maoni yake juu ya maendeleo ya hivi karibuni ya AI katika sayansi. Pia tumepokea maoni kutoka kwa wanabiolojia wa kompyuta na wahandisi wa kibiolojia. "Demes Hassabis, Mkurugenzi Mtendaji wa Google DeepMind, ameshinda tuzo ya Lasker ya 2023 katika Sayansi ya Msingi ya Matibabu kwa mchango wake kwa AlphaFold, mfumo wa AI unaotabiri muundo wa tatu wa protini kutoka kwa mfululizo wa asidi amino. """ "Mfano wa ""maelezo ya lugha"" ni kwamba, kwa mfano, lugha inaweza kutumika kwa kuita moja au zaidi ya zana za AI, kama vile AlphaFold, baada ya kupokea ombi kutoka kwa mtumiaji." "Kifaa kingine cha AI kilichoundwa na kikundi chake, kilichochapishwa katika Sayansi mnamo Septemba 2023, ni ""AlphaMissSense,"" ambayo inabashiri pathogenicity ya mabadiliko yote ya amino asidi ya binadamu." Vipengele vyote vya AlphaFold AI vilitumiwa kwa wazi, lakini katika kesi ya AlphaMissSense, uzito uliozoezwa, seti ya vigezo muhimu kwa kuendesha mfano, haikutumiwa. Uamuzi huu na mwingine kama huo uliopitishwa kwa LLM kadhaa ni mfano wa mapungufu ya sayansi ya wazi ambayo yanaibuka katika uwanja wa AI, hata wakati AI hutumiwa kwa sayansi ya msingi. Matokeo ya utafiti (Oktoba 2023) - wataalamu wa uchumi Uchunguzi wa wataalamu wa uchumi ulipokea majibu 33. Zaidi ya asilimia 62 ya washiriki wa utafiti huo ni wanachama wa NBER (National Bureau of Economic Research), moja ya makundi ya wataalamu wa uchumi. Hizi ni hasa wasomi mashuhuri kutoka vyuo vikuu vya Amerika Kaskazini. Pia, tumepokea majibu kutoka kwa kikundi cha kimataifa cha wachumi wanaohusishwa na vyuo vikuu nchini Marekani, Canada, Uingereza, Norway, Italia, Denmark, India, Japan, New Zealand na Benki ya Dunia na IMF. "Tuliuliza maswali yale yale ambayo yalikuwa yamejibiwa na wanasayansi wa biomedical, sasa yameelekezwa kwa wanasayansi kwa ujumla, na maswali machache ya ziada kwa wachumi: ""Mifumo ya akili bandia (AI) ina kiasi kikubwa na kuongezeka kwa maarifa ya msingi kuhusu sayansi.""" Kuingiliana na mifumo ya AI inachangia mafunzo yao; mfano wa hii ni kujifunza kuimarisha kutoka kwa maoni ya binadamu. Maswali ya jumla kwa wanasayansi wote: Je, unafikiri ni muhimu kwa jamii ya kisayansi kujadili udhibiti wa mifumo ya AI? Q2 - Q2 Ni nini faida na hasara za kukuza maendeleo ya mifumo ya AI ambayo ni wazi zaidi, na ambapo jamii ya kisayansi inashiriki katika udhibiti? Q3 - Uwezo wa kuamua Je, wanasayansi wanapaswa kuingiliana kwa upendeleo na mifumo ya AI ambayo ina sifa hizi (ufunuo zaidi na ushiriki wa kudhibiti)? Tangu mwanzo wa mapinduzi ya kisayansi katika karne ya kumi na saba, maarifa ya msingi ya kisayansi ya asili imekuwa rasilimali ya umma, wakati maarifa ya matumizi ya thamani ya kibiashara ama kulindwa na hati miliki, ambayo zinahitaji kufichuliwa kwa kubadilishana na upendeleo wa muda, au kuhifadhiwa kama siri. Mifumo kadhaa kubwa ya AI muhimu kwa sayansi ya msingi na ya kutumika, pamoja na Mifano Kubwa ya Lugha (LLMs) na zana zingine za AI, sasa zinaendelezwa na makampuni machache ambayo huhifadhi vigezo muhimu vya kiasi kama siri ya wamiliki. Mifumo hii ya AI inahitaji uwekezaji mkubwa. Q4 - Q4 Jinsi gani kuibuka kwa mifumo hii ya kibinafsi ya AI inaweza kuathiri mtiririko wa mawazo? Hii inamaanisha nini kuhusu data na uadilifu wa kisayansi katika siku zijazo? Jambo hilo linamaanisha nini kuhusu mwendo na usambazaji wa ukuzi wa kiuchumi? Q5 - Uwezo wa kuamua Ni jukumu gani la umma katika kudhibiti mifano hii ya AI, ili kuongeza faida zao na kupunguza hatari zao? Q6: Ni nini itakuwa matokeo ya kiuchumi kama mifumo ya umma ya AI, iliyojitolea kwa maarifa ya msingi ya kisayansi, ingekuwa na ushirikiano na juhudi za kibinafsi za AI, zilizolenga matumizi maalum? Wataalamu wote wa uchumi waliojibu ndiyo kwa swali la 1. Unaweza kusoma muhtasari wa majibu na majibu yote ya mtu binafsi. Kwa mfano, AI (ChatGPT 3.5) ilitumiwa kuunda muhtasari wa utafiti na kuhariri kwa binadamu. 1-Ghasemi M, Oakden-Rayner L, Beam AL. "Tumaini bandia la mbinu za sasa za kueleza akili bandia katika utunzaji wa afya". The Lancet Digital Afya 2021 Nov 1;3 (11):e745-50 "Burnell R., Schellert W., Burden J., Ullman TD., Martinez-Plumed F., Tenenbaum J.B., Ruter D., Cheke L., Sohl-Dickenstein J., Mitchell M., Kiela D., ""Rethink reporting of evaluation results in AI.""" Sayansi. Aprili 14, 2021 - 386: 1 - 8 "Michel M. Mitchell: ""Nini kinachotufanya tuwe na akili zaidi?""" Sayansi. 2023 Jul 13;381(6654):adj5957. "Hii ni ""Historia ya ndani ya jinsi ChatGPT ilivyotengenezwa kutoka kwa watu walioiunda.""" MIT Technology Review 2023 - Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A, Schulman J. "Kufundisha mifano ya lugha kufuata maagizo na maoni ya kibinadamu". Maendeleo katika Neural Information Processing Systems. Dec. 6, 2020 (Siku ya Mkutano wa Wenyeji wa Nchi) 6 Thirunavukarasu AJ, Ting DS, Elangovan K, Gutierrez L, Tan TF, Ting DS. "Mifano mikubwa ya lugha katika tiba". Dawa ya Asili. "Julai 17, 2021 - Taddeo M. Floridi, L. ""Jinsi AI inaweza kuwa nguvu kwa ajili ya mema.""" Sayansi. 2018 Aug 24;361(6404):751-2.