Artificial intelligence and illusions of understanding in scientific research
Artificial intelligence and illusions of understanding in scientific research
Crabtree, G. Self-driving laboratories coming of age. Joule 4, 2538–2541 (2020).Article 
    CAS 
    
                    Google Scholar 
                Wang, H. et al. Scientific discovery in the age of artificial intelligence. Nature 620, 47–60 (2023). This review explores how AI can be incorporated across the research pipeline, drawing from a wide range of scientific disciplines.Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Dillion, D., Tandon, N., Gu, Y. & Gray, K. Can AI language models replace human participants? Trends Cogn. Sci. 27, 597–600 (2023).Article 
    PubMed 
    
                    Google Scholar 
                Grossmann, I. et al. AI and the transformation of social science research. Science 380, 1108–1109 (2023). This forward-looking article proposes a variety of ways to incorporate generative AI into social-sciences research.Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Gil, Y. Will AI write scientific papers in the future? AI Mag. 42, 3–15 (2022).
                    Google Scholar 
                Kitano, H. Nobel Turing Challenge: creating the engine for scientific discovery. npj Syst. Biol. Appl. 7, 29 (2021).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Benjamin, R. Race After Technology: Abolitionist Tools for the New Jim Code (Oxford Univ. Press, 2020). This book examines how social norms about race become embedded in technologies, even those that are focused on providing good societal outcomes.Broussard, M. More Than a Glitch: Confronting Race, Gender, and Ability Bias in Tech (MIT Press, 2023).Noble, S. U. Algorithms of Oppression: How Search Engines Reinforce Racism (New York Univ. Press, 2018).Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. On the dangers of stochastic parrots: can language models be too big? in Proc. 2021 ACM Conference on Fairness, Accountability, and Transparency 610–623 (Association for Computing Machinery, 2021). One of the first comprehensive critiques of large language models, this article draws attention to a host of issues that ought to be considered before taking up such tools.Crawford, K. Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence (Yale Univ. Press, 2021).Johnson, D. G. & Verdicchio, M. Reframing AI discourse. Minds Mach. 27, 575–590 (2017).Article 
    
                    Google Scholar 
                Atanasoski, N. & Vora, K. Surrogate Humanity: Race, Robots, and the Politics of Technological Futures (Duke Univ. Press, 2019).Mitchell, M. & Krakauer, D. C. The debate over understanding in AI’s large language models. Proc. Natl Acad. Sci. USA 120, e2215907120 (2023).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Kidd, C. & Birhane, A. How AI can distort human beliefs. Science 380, 1222–1223 (2023).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Birhane, A., Kasirzadeh, A., Leslie, D. & Wachter, S. Science in the age of large language models. Nat. Rev. Phys. 5, 277–280 (2023).Article 
    
                    Google Scholar 
                Kapoor, S. & Narayanan, A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns 4, 100804 (2023).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Hullman, J., Kapoor, S., Nanayakkara, P., Gelman, A. & Narayanan, A. The worst of both worlds: a comparative analysis of errors in learning from data in psychology and machine learning. In Proc. 2022 AAAI/ACM Conference on AI, Ethics, and Society (eds Conitzer, V. et al.) 335–348 (Association for Computing Machinery, 2022).Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019). This paper articulates the problems with attempting to explain AI systems that lack interpretability, and advocates for building interpretable models instead.Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Crockett, M. J., Bai, X., Kapoor, S., Messeri, L. & Narayanan, A. The limitations of machine learning models for predicting scientific replicability. Proc. Natl Acad. Sci. USA 120, e2307596120 (2023).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Lazar, S. & Nelson, A. AI safety on whose terms? Science 381, 138 (2023).Article 
    PubMed 
    ADS 
    
                    Google Scholar 
                Collingridge, D. The Social Control of Technology (St Martin’s Press, 1980).Wagner, G., Lukyanenko, R. & Paré, G. Artificial intelligence and the conduct of literature reviews. J. Inf. Technol. 37, 209–226 (2022).Article 
    
                    Google Scholar 
                Hutson, M. Artificial-intelligence tools aim to tame the coronavirus literature. Nature https://doi.org/10.1038/d41586-020-01733-7 (2020).Article 
    PubMed 
    
                    Google Scholar 
                Haas, Q. et al. Utilizing artificial intelligence to manage COVID-19 scientific evidence torrent with Risklick AI: a critical tool for pharmacology and therapy development. Pharmacology 106, 244–253 (2021).Article 
    CAS 
    PubMed 
    
                    Google Scholar 
                Müller, H., Pachnanda, S., Pahl, F. & Rosenqvist, C. The application of artificial intelligence on different types of literature reviews – a comparative study. In 2022 International Conference on Applied Artificial Intelligence (ICAPAI) https://doi.org/10.1109/ICAPAI55158.2022.9801564 (Institute of Electrical and Electronics Engineers, 2022).van Dinter, R., Tekinerdogan, B. & Catal, C. Automation of systematic literature reviews: a systematic literature review. Inf. Softw. Technol. 136, 106589 (2021).Article 
    
                    Google Scholar 
                Aydın, Ö. & Karaarslan, E. OpenAI ChatGPT generated literature review: digital twin in healthcare. In Emerging Computer Technologies 2 (ed. Aydın, Ö.) 22–31 (İzmir Akademi Dernegi, 2022).AlQuraishi, M. AlphaFold at CASP13. Bioinformatics 35, 4862–4865 (2019).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Lee, J. S., Kim, J. & Kim, P. M. Score-based generative modeling for de novo protein design. Nat. Computat. Sci. 3, 382–392 (2023).Article 
    CAS 
    
                    Google Scholar 
                Gómez-Bombarelli, R. et al. Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach. Nat. Mater. 15, 1120–1127 (2016).Article 
    PubMed 
    ADS 
    
                    Google Scholar 
                Krenn, M. et al. On scientific understanding with artificial intelligence. Nat. Rev. Phys. 4, 761–769 (2022).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Extance, A. How AI technology can tame the scientific literature. Nature 561, 273–274 (2018).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Hastings, J. AI for Scientific Discovery (CRC Press, 2023). This book reviews current and future incorporation of AI into the scientific research pipeline.Ahmed, A. et al. The future of academic publishing. Nat. Hum. Behav. 7, 1021–1026 (2023).Article 
    PubMed 
    
                    Google Scholar 
                Gray, K., Yam, K. C., Zhen’An, A. E., Wilbanks, D. & Waytz, A. The psychology of robots and artificial intelligence. In The Handbook of Social Psychology (eds Gilbert, D. et al.) (in the press).Argyle, L. P. et al. Out of one, many: using language models to simulate human samples. Polit. Anal. 31, 337–351 (2023).Article 
    
                    Google Scholar 
                Aher, G., Arriaga, R. I. & Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In Proc. 40th International Conference on Machine Learning (eds Krause, A. et al.) 337–371 (JMLR.org, 2023).Binz, M. & Schulz, E. Using cognitive psychology to understand GPT-3. Proc. Natl Acad. Sci. USA 120, e2218523120 (2023).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Ornstein, J. T., Blasingame, E. N. & Truscott, J. S. How to train your stochastic parrot: large language models for political texts. Github, https://joeornstein.github.io/publications/ornstein-blasingame-truscott.pdf (2023).He, S. et al. Learning to predict the cosmological structure formation. Proc. Natl Acad. Sci. USA 116, 13825–13832 (2019).Article 
    MathSciNet 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Mahmood, F. et al. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE Trans. Med. Imaging 39, 3257–3267 (2020).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Teixeira, B. et al. Generating synthetic X-ray images of a person from the surface geometry. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 9059–9067 (Institute of Electrical and Electronics Engineers, 2018).Marouf, M. et al. Realistic in silico generation and augmentation of single-cell RNA-seq data using generative adversarial networks. Nat. Commun. 11, 166 (2020).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Watts, D. J. A twenty-first century science. Nature 445, 489 (2007).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                boyd, d. & Crawford, K. Critical questions for big data. Inf. Commun. Soc. 15, 662–679 (2012). This article assesses the ethical and epistemic implications of scientific and societal moves towards big data and provides a parallel case study for thinking about the risks of artificial intelligence.Article 
    
                    Google Scholar 
                Jolly, E. & Chang, L. J. The Flatland fallacy: moving beyond low–dimensional thinking. Top. Cogn. Sci. 11, 433–454 (2019).Article 
    PubMed 
    
                    Google Scholar 
                Yarkoni, T. & Westfall, J. Choosing prediction over explanation in psychology: lessons from machine learning. Perspect. Psychol. Sci. 12, 1100–1122 (2017).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Radivojac, P. et al. A large-scale evaluation of computational protein function prediction. Nat. Methods 10, 221–227 (2013).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Bileschi, M. L. et al. Using deep learning to annotate the protein universe. Nat. Biotechnol. 40, 932–937 (2022).Article 
    CAS 
    PubMed 
    
                    Google Scholar 
                Barkas, N. et al. Joint analysis of heterogeneous single-cell RNA-seq dataset collections. Nat. Methods 16, 695–698 (2019).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Demszky, D. et al. Using large language models in psychology. Nat. Rev. Psychol. 2, 688–701 (2023).Article 
    
                    Google Scholar 
                Karjus, A. Machine-assisted mixed methods: augmenting humanities and social sciences with artificial intelligence. Preprint at https://arxiv.org/abs/2309.14379 (2023).Davies, A. et al. Advancing mathematics by guiding human intuition with AI. Nature 600, 70–74 (2021).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Peterson, J. C., Bourgin, D. D., Agrawal, M., Reichman, D. & Griffiths, T. L. Using large-scale experiments and machine learning to discover theories of human decision-making. Science 372, 1209–1214 (2021).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Ilyas, A. et al. Adversarial examples are not bugs, they are features. Preprint at https://doi.org/10.48550/arXiv.1905.02175 (2019)Semel, B. M. Listening like a computer: attentional tensions and mechanized care in psychiatric digital phenotyping. Sci. Technol. Hum. Values 47, 266–290 (2022).Article 
    
                    Google Scholar 
                Gil, Y. Thoughtful artificial intelligence: forging a new partnership for data science and scientific discovery. Data Sci. 1, 119–129 (2017).Article 
    
                    Google Scholar 
                Checco, A., Bracciale, L., Loreti, P., Pinfield, S. & Bianchi, G. AI-assisted peer review. Humanit. Soc. Sci. Commun. 8, 25 (2021).Article 
    
                    Google Scholar 
                Thelwall, M. Can the quality of published academic journal articles be assessed with machine learning? Quant. Sci. Stud. 3, 208–226 (2022).Article 
    
                    Google Scholar 
                Dhar, P. Peer review of scholarly research gets an AI boost. IEEE Spectrum spectrum.ieee.org/peer-review-of-scholarly-research-gets-an-ai-boost (2020).Heaven, D. AI peer reviewers unleashed to ease publishing grind. Nature 563, 609–610 (2018).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Conroy, G. How ChatGPT and other AI tools could disrupt scientific publishing. Nature 622, 234–236 (2023).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Nosek, B. A. et al. Replicability, robustness, and reproducibility in psychological science. Annu. Rev. Psychol. 73, 719–748 (2022).Article 
    PubMed 
    
                    Google Scholar 
                Altmejd, A. et al. Predicting the replicability of social science lab experiments. PLoS ONE 14, e0225826 (2019).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Yang, Y., Youyou, W. & Uzzi, B. Estimating the deep replicability of scientific findings using human and artificial intelligence. Proc. Natl Acad. Sci. USA 117, 10762–10768 (2020).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Youyou, W., Yang, Y. & Uzzi, B. A discipline-wide investigation of the replicability of psychology papers over the past two decades. Proc. Natl Acad. Sci. USA 120, e2208863120 (2023).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Rabb, N., Fernbach, P. M. & Sloman, S. A. Individual representation in a community of knowledge. Trends Cogn. Sci. 23, 891–902 (2019). This comprehensive review paper documents the empirical evidence for distributed cognition in communities of knowledge and the resultant vulnerabilities to illusions of understanding.Article 
    PubMed 
    
                    Google Scholar 
                Rozenblit, L. & Keil, F. The misunderstood limits of folk science: an illusion of explanatory depth. Cogn. Sci. 26, 521–562 (2002). This paper provided an empirical demonstration of the illusion of explanatory depth, and inspired a programme of research in cognitive science on communities of knowledge.Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Hutchins, E. Cognition in the Wild (MIT Press, 1995).Lave, J. & Wenger, E. Situated Learning: Legitimate Peripheral Participation (Cambridge Univ. Press, 1991).Kitcher, P. The division of cognitive labor. J. Philos. 87, 5–22 (1990).Article 
    
                    Google Scholar 
                Hardwig, J. Epistemic dependence. J. Philos. 82, 335–349 (1985).Article 
    
                    Google Scholar 
                Keil, F. in Oxford Studies In Epistemology (eds Gendler, T. S. & Hawthorne, J.) 143–166 (Oxford Academic, 2005).Weisberg, M. & Muldoon, R. Epistemic landscapes and the division of cognitive labor. Philos. Sci. 76, 225–252 (2009).Article 
    
                    Google Scholar 
                Sloman, S. A. & Rabb, N. Your understanding is my understanding: evidence for a community of knowledge. Psychol. Sci. 27, 1451–1460 (2016).Article 
    PubMed 
    
                    Google Scholar 
                Wilson, R. A. & Keil, F. The shadows and shallows of explanation. Minds Mach. 8, 137–159 (1998).Article 
    
                    Google Scholar 
                Keil, F. C., Stein, C., Webb, L., Billings, V. D. & Rozenblit, L. Discerning the division of cognitive labor: an emerging understanding of how knowledge is clustered in other minds. Cogn. Sci. 32, 259–300 (2008).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Sperber, D. et al. Epistemic vigilance. Mind Lang. 25, 359–393 (2010).Article 
    
                    Google Scholar 
                Wilkenfeld, D. A., Plunkett, D. & Lombrozo, T. Depth and deference: when and why we attribute understanding. Philos. Stud. 173, 373–393 (2016).Article 
    
                    Google Scholar 
                Sparrow, B., Liu, J. & Wegner, D. M. Google effects on memory: cognitive consequences of having information at our fingertips. Science 333, 776–778 (2011).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Fisher, M., Goddu, M. K. & Keil, F. C. Searching for explanations: how the internet inflates estimates of internal knowledge. J. Exp. Psychol. Gen. 144, 674–687 (2015).Article 
    PubMed 
    
                    Google Scholar 
                De Freitas, J., Agarwal, S., Schmitt, B. & Haslam, N. Psychological factors underlying attitudes toward AI tools. Nat. Hum. Behav. 7, 1845–1854 (2023).Article 
    PubMed 
    
                    Google Scholar 
                Castelo, N., Bos, M. W. & Lehmann, D. R. Task-dependent algorithm aversion. J. Mark. Res. 56, 809–825 (2019).Article 
    
                    Google Scholar 
                Cadario, R., Longoni, C. & Morewedge, C. K. Understanding, explaining, and utilizing medical artificial intelligence. Nat. Hum. Behav. 5, 1636–1642 (2021).Article 
    PubMed 
    
                    Google Scholar 
                Oktar, K. & Lombrozo, T. Deciding to be authentic: intuition is favored over deliberation when authenticity matters. Cognition 223, 105021 (2022).Article 
    PubMed 
    
                    Google Scholar 
                Bigman, Y. E., Yam, K. C., Marciano, D., Reynolds, S. J. & Gray, K. Threat of racial and economic inequality increases preference for algorithm decision-making. Comput. Hum. Behav. 122, 106859 (2021).Article 
    
                    Google Scholar 
                Claudy, M. C., Aquino, K. & Graso, M. Artificial intelligence can’t be charmed: the effects of impartiality on laypeople’s algorithmic preferences. Front. Psychol. 13, 898027 (2022).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Snyder, C., Keppler, S. & Leider, S. Algorithm reliance under pressure: the effect of customer load on service workers. Preprint at SSRN https://doi.org/10.2139/ssrn.4066823 (2022).Bogert, E., Schecter, A. & Watson, R. T. Humans rely more on algorithms than social influence as a task becomes more difficult. Sci Rep. 11, 8028 (2021).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Raviv, A., Bar‐Tal, D., Raviv, A. & Abin, R. Measuring epistemic authority: studies of politicians and professors. Eur. J. Personal. 7, 119–138 (1993).Article 
    
                    Google Scholar 
                Cummings, L. The “trust” heuristic: arguments from authority in public health. Health Commun. 29, 1043–1056 (2014).Article 
    PubMed 
    ADS 
    
                    Google Scholar 
                Lee, M. K. Understanding perception of algorithmic decisions: fairness, trust, and emotion in response to algorithmic management. Big Data Soc. 5, https://doi.org/10.1177/2053951718756684 (2018).Kissinger, H. A., Schmidt, E. & Huttenlocher, D. The Age of A.I. And Our Human Future (Little, Brown, 2021).Lombrozo, T. Explanatory preferences shape learning and inference. Trends Cogn. Sci. 20, 748–759 (2016). This paper provides an overview of philosophical theories of explanatory virtues and reviews empirical evidence on the sorts of explanations people find satisfying.Article 
    PubMed 
    
                    Google Scholar 
                Vrantsidis, T. H. & Lombrozo, T. Simplicity as a cue to probability: multiple roles for simplicity in evaluating explanations. Cogn. Sci. 46, e13169 (2022).Article 
    PubMed 
    
                    Google Scholar 
                Johnson, S. G. B., Johnston, A. M., Toig, A. E. & Keil, F. C. Explanatory scope informs causal strength inferences. In Proc. 36th Annual Meeting of the Cognitive Science Society 2453–2458 (Cognitive Science Society, 2014).Khemlani, S. S., Sussman, A. B. & Oppenheimer, D. M. Harry Potter and the sorcerer’s scope: latent scope biases in explanatory reasoning. Mem. Cognit. 39, 527–535 (2011).Article 
    PubMed 
    
                    Google Scholar 
                Liquin, E. G. & Lombrozo, T. Motivated to learn: an account of explanatory satisfaction. Cogn. Psychol. 132, 101453 (2022).Article 
    PubMed 
    
                    Google Scholar 
                Hopkins, E. J., Weisberg, D. S. & Taylor, J. C. V. The seductive allure is a reductive allure: people prefer scientific explanations that contain logically irrelevant reductive information. Cognition 155, 67–76 (2016).Article 
    PubMed 
    
                    Google Scholar 
                Weisberg, D. S., Hopkins, E. J. & Taylor, J. C. V. People’s explanatory preferences for scientific phenomena. Cogn. Res. Princ. Implic. 3, 44 (2018).Article 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Jerez-Fernandez, A., Angulo, A. N. & Oppenheimer, D. M. Show me the numbers: precision as a cue to others’ confidence. Psychol. Sci. 25, 633–635 (2014).Article 
    PubMed 
    
                    Google Scholar 
                Kim, J., Giroux, M. & Lee, J. C. When do you trust AI? The effect of number presentation detail on consumer trust and acceptance of AI recommendations. Psychol. Mark. 38, 1140–1155 (2021).Article 
    
                    Google Scholar 
                Nguyen, C. T. The seductions of clarity. R. Inst. Philos. Suppl. 89, 227–255 (2021). This article describes how reductive and quantitative explanations can generate a sense of understanding that is not necessarily correlated with actual understanding.Article 
    
                    Google Scholar 
                Fisher, M., Smiley, A. H. & Grillo, T. L. H. Information without knowledge: the effects of internet search on learning. Memory 30, 375–387 (2022).Article 
    
                    Google Scholar 
                Eliseev, E. D. & Marsh, E. J. Understanding why searching the internet inflates confidence in explanatory ability. Appl. Cogn. Psychol. 37, 711–720 (2023).Article 
    
                    Google Scholar 
                Fisher, M. & Oppenheimer, D. M. Who knows what? Knowledge misattribution in the division of cognitive labor. J. Exp. Psychol. Appl. 27, 292–306 (2021).Article 
    PubMed 
    
                    Google Scholar 
                Chromik, M., Eiband, M., Buchner, F., Krüger, A. & Butz, A. I think I get your point, AI! The illusion of explanatory depth in explainable AI. In 26th International Conference on Intelligent User Interfaces (eds Hammond, T. et al.) 307–317 (Association for Computing Machinery, 2021).Strevens, M. No understanding without explanation. Stud. Hist. Philos. Sci. A 44, 510–515 (2013).Article 
    
                    Google Scholar 
                Ylikoski, P. in Scientific Understanding: Philosophical Perspectives (eds De Regt, H. et al.) 100–119 (Univ. Pittsburgh Press, 2009).Giudice, M. D. The prediction–explanation fallacy: a pervasive problem in scientific applications of machine learning. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/4vq8f (2021).Hofman, J. M. et al. Integrating explanation and prediction in computational social science. Nature 595, 181–188 (2021). This paper highlights the advantages and disadvantages of explanatory versus predictive approaches to modelling, with a focus on applications to computational social science.Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Shmueli, G. To explain or to predict? Stat. Sci. 25, 289–310 (2010).Article 
    MathSciNet 
    
                    Google Scholar 
                Hofman, J. M., Sharma, A. & Watts, D. J. Prediction and explanation in social systems. Science 355, 486–488 (2017).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Logg, J. M., Minson, J. A. & Moore, D. A. Algorithm appreciation: people prefer algorithmic to human judgment. Organ. Behav. Hum. Decis. Process. 151, 90–103 (2019).Article 
    
                    Google Scholar 
                Nguyen, C. T. Cognitive islands and runaway echo chambers: problems for epistemic dependence on experts. Synthese 197, 2803–2821 (2020).Article 
    
                    Google Scholar 
                Breiman, L. Statistical modeling: the two cultures. Stat. Sci. 16, 199–215 (2001).Article 
    MathSciNet 
    
                    Google Scholar 
                Gao, J. & Wang, D. Quantifying the benefit of artificial intelligence for scientific research. Preprint at arxiv.org/abs/2304.10578 (2023).Hanson, B. et al. Garbage in, garbage out: mitigating risks and maximizing benefits of AI in research. Nature 623, 28–31 (2023).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Kleinberg, J. & Raghavan, M. Algorithmic monoculture and social welfare. Proc. Natl Acad. Sci. USA 118, e2018340118 (2021). This paper uses formal modelling methods to demonstrate that when companies all rely on the same algorithm to make decisions (an algorithmic monoculture), the overall quality of those decisions is reduced because valuable options can slip through the cracks, even when the algorithm performs accurately for individual companies.Article 
    MathSciNet 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Hofstra, B. et al. The diversity–innovation paradox in science. Proc. Natl Acad. Sci. USA 117, 9284–9291 (2020).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Hong, L. & Page, S. E. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proc. Natl Acad. Sci. USA 101, 16385–16389 (2004).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Page, S. E. Where diversity comes from and why it matters? Eur. J. Soc. Psychol. 44, 267–279 (2014). This article reviews research demonstrating the benefits of cognitive diversity and diversity in methodological approaches for problem solving and innovation.Article 
    
                    Google Scholar 
                Clarke, A. E. & Fujimura, J. H. (eds) The Right Tools for the Job: At Work in Twentieth-Century Life Sciences (Princeton Univ. Press, 2014).Silva, V. J., Bonacelli, M. B. M. & Pacheco, C. A. Framing the effects of machine learning on science. AI Soc. https://doi.org/10.1007/s00146-022-01515-x (2022).Sassenberg, K. & Ditrich, L. Research in social psychology changed between 2011 and 2016: larger sample sizes, more self-report measures, and more online studies. Adv. Methods Pract. Psychol. Sci. 2, 107–114 (2019).Article 
    
                    Google Scholar 
                Simon, A. F. & Wilder, D. Methods and measures in social and personality psychology: a comparison of JPSP publications in 1982 and 2016. J. Soc. Psychol. https://doi.org/10.1080/00224545.2022.2135088 (2022).Article 
    PubMed 
    
                    Google Scholar 
                Anderson, C. A. et al. The MTurkification of social and personality psychology. Pers. Soc. Psychol. Bull. 45, 842–850 (2019).Article 
    PubMed 
    
                    Google Scholar 
                Latour, B. in The Social After Gabriel Tarde: Debates and Assessments (ed. Candea, M.) 145–162 (Routledge, 2010).Porter, T. M. Trust in Numbers: The Pursuit of Objectivity in Science and Public Life (Princeton Univ. Press, 1996).Lazer, D. et al. Meaningful measures of human society in the twenty-first century. Nature 595, 189–196 (2021).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Knox, D., Lucas, C. & Cho, W. K. T. Testing causal theories with learned proxies. Annu. Rev. Polit. Sci. 25, 419–441 (2022).Article 
    
                    Google Scholar 
                Barberá, P. Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data. Polit. Anal. 23, 76–91 (2015).Article 
    
                    Google Scholar 
                Brady, W. J., McLoughlin, K., Doan, T. N. & Crockett, M. J. How social learning amplifies moral outrage expression in online social networks. Sci. Adv. 7, eabe5641 (2021).Article 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Barnes, J., Klinger, R. & im Walde, S. S. Assessing state-of-the-art sentiment models on state-of-the-art sentiment datasets. In Proc. 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (eds Balahur, A. et al.) 2–12 (Association for Computational Linguistics, 2017).Gitelman, L. (ed.) “Raw Data” is an Oxymoron (MIT Press, 2013).Breznau, N. et al. Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proc. Natl Acad. Sci. USA 119, e2203150119 (2022). This study demonstrates how 73 research teams analysing the same dataset reached different conclusions about the relationship between immigration and public support for social policies, highlighting the subjectivity and uncertainty involved in analysing complex datasets.Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Gillespie, T. in Media Technologies: Essays on Communication, Materiality, and Society (eds Gillespie, T. et al.) 167–194 (MIT Press, 2014).Leonelli, S. Data-Centric Biology: A Philosophical Study (Univ. Chicago Press, 2016).Wang, A., Kapoor, S., Barocas, S. & Narayanan, A. Against predictive optimization: on the legitimacy of decision-making algorithms that optimize predictive accuracy. ACM J. Responsib. Comput., https://doi.org/10.1145/3636509 (2023).Athey, S. Beyond prediction: using big data for policy problems. Science 355, 483–485 (2017).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                del Rosario Martínez-Ordaz, R. Scientific understanding through big data: from ignorance to insights to understanding. Possibility Stud. Soc. 1, 279–299 (2023).Article 
    
                    Google Scholar 
                Nussberger, A.-M., Luo, L., Celis, L. E. & Crockett, M. J. Public attitudes value interpretability but prioritize accuracy in artificial intelligence. Nat. Commun. 13, 5821 (2022).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Zittrain, J. in The Cambridge Handbook of Responsible Artificial Intelligence: Interdisciplinary Perspectives (eds. Voeneky, S. et al.) 176–184 (Cambridge Univ. Press, 2022). This article articulates the epistemic risks of prioritizing predictive accuracy over explanatory understanding when AI tools are interacting in complex systems.Shumailov, I. et al. The curse of recursion: training on generated data makes models forget. Preprint at arxiv.org/abs/2305.17493 (2023).Latour, B. Science In Action: How to Follow Scientists and Engineers Through Society (Harvard Univ. Press, 1987). This book provides strategies and approaches for thinking about science as a social endeavour.Franklin, S. Science as culture, cultures of science. Annu. Rev. Anthropol. 24, 163–184 (1995).Article 
    
                    Google Scholar 
                Haraway, D. Situated knowledges: the science question in feminism and the privilege of partial perspective. Fem. Stud. 14, 575–599 (1988). This article acknowledges that the objective ‘view from nowhere’ is unobtainable: knowledge, it argues, is always situated.Article 
    
                    Google Scholar 
                Harding, S. Objectivity and Diversity: Another Logic of Scientific Research (Univ. Chicago Press, 2015).Longino, H. E. Science as Social Knowledge: Values and Objectivity in Scientific Inquiry (Princeton Univ. Press, 1990).Daston, L. & Galison, P. Objectivity (Princeton Univ. Press, 2007). This book is a historical analysis of the shifting modes of ‘objectivity’ that scientists have pursued, arguing that objectivity is not a universal concept but that it shifts alongside scientific techniques and ambitions.Prescod-Weinstein, C. Making Black women scientists under white empiricism: the racialization of epistemology in physics. Signs J. Women Cult. Soc. 45, 421–447 (2020).Article 
    
                    Google Scholar 
                Mavhunga, C. What Do Science, Technology, and Innovation Mean From Africa? (MIT Press, 2017).Schiebinger, L. The Mind Has No Sex? Women in the Origins of Modern Science (Harvard Univ. Press, 1991).Martin, E. The egg and the sperm: how science has constructed a romance based on stereotypical male–female roles. Signs J. Women Cult. Soc. 16, 485–501 (1991). This case study shows how assumptions about gender affect scientific theories, sometimes delaying the articulation of what might be considered to be more accurate descriptions of scientific phenomena.Article 
    
                    Google Scholar 
                Harding, S. Rethinking standpoint epistemology: What is “strong objectivity”? Centen. Rev. 36, 437–470 (1992). In this article, Harding outlines her position on ‘strong objectivity’, by which clearly articulating one’s standpoint can lead to more robust knowledge claims.
                    Google Scholar 
                Oreskes, N. Why Trust Science? (Princeton Univ. Press, 2019). This book introduces the reader to 20 years of scholarship in science and technology studies, arguing that the tools the discipline has for understanding science can help to reinstate public trust in the institution.Rolin, K., Koskinen, I., Kuorikoski, J. & Reijula, S. Social and cognitive diversity in science: introduction. Synthese 202, 36 (2023).Article 
    
                    Google Scholar 
                Hong, L. & Page, S. E. Problem solving by heterogeneous agents. J. Econ. Theory 97, 123–163 (2001).Article 
    MathSciNet 
    
                    Google Scholar 
                Sulik, J., Bahrami, B. & Deroy, O. The diversity gap: when diversity matters for knowledge. Perspect. Psychol. Sci. 17, 752–767 (2022).Article 
    PubMed 
    
                    Google Scholar 
                Lungeanu, A., Whalen, R., Wu, Y. J., DeChurch, L. A. & Contractor, N. S. Diversity, networks, and innovation: a text analytic approach to measuring expertise diversity. Netw. Sci. 11, 36–64 (2023).Article 
    
                    Google Scholar 
                AlShebli, B. K., Rahwan, T. & Woon, W. L. The preeminence of ethnic diversity in scientific collaboration. Nat. Commun. 9, 5163 (2018).Article 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Campbell, L. G., Mehtani, S., Dozier, M. E. & Rinehart, J. Gender-heterogeneous working groups produce higher quality science. PLoS ONE 8, e79147 (2013).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Nielsen, M. W., Bloch, C. W. & Schiebinger, L. Making gender diversity work for scientific discovery and innovation. Nat. Hum. Behav. 2, 726–734 (2018).Article 
    PubMed 
    
                    Google Scholar 
                Yang, Y., Tian, T. Y., Woodruff, T. K., Jones, B. F. & Uzzi, B. Gender-diverse teams produce more novel and higher-impact scientific ideas. Proc. Natl Acad. Sci. USA 119, e2200841119 (2022).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Kozlowski, D., Larivière, V., Sugimoto, C. R. & Monroe-White, T. Intersectional inequalities in science. Proc. Natl Acad. Sci. USA 119, e2113067119 (2022).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Fehr, C. & Jones, J. M. Culture, exploitation, and epistemic approaches to diversity. Synthese 200, 465 (2022).Article 
    MathSciNet 
    
                    Google Scholar 
                Nakadai, R., Nakawake, Y. & Shibasaki, S. AI language tools risk scientific diversity and innovation. Nat. Hum. Behav. 7, 1804–1805 (2023).Article 
    PubMed 
    
                    Google Scholar 
                National Academies of Sciences, Engineering, and Medicine et al. Advancing Antiracism, Diversity, Equity, and Inclusion in STEMM Organizations: Beyond Broadening Participation (National Academies Press, 2023).Winner, L. Do artifacts have politics? Daedalus 109, 121–136 (1980).
                    Google Scholar 
                Eubanks, V. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (St. Martin’s Press, 2018).Littmann, M. et al. Validity of machine learning in biology and medicine increased through collaborations across fields of expertise. Nat. Mach. Intell. 2, 18–24 (2020).Article 
    
                    Google Scholar 
                Carusi, A. et al. Medical artificial intelligence is as much social as it is technological. Nat. Mach. Intell. 5, 98–100 (2023).Article 
    
                    Google Scholar 
                Raghu, M. & Schmidt, E. A survey of deep learning for scientific discovery. Preprint at arxiv.org/abs/2003.11755 (2020).Bishop, C. AI4Science to empower the fifth paradigm of scientific discovery. Microsoft Research Blog www.microsoft.com/en-us/research/blog/ai4science-to-empower-the-fifth-paradigm-of-scientific-discovery/ (2022).Whittaker, M. The steep cost of capture. Interactions 28, 50–55 (2021).Article 
    
                    Google Scholar 
                Liesenfeld, A., Lopez, A. & Dingemanse, M. Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators. In Proc. 5th International Conference on Conversational User Interfaces 1–6 (Association for Computing Machinery, 2023).Chu, J. S. G. & Evans, J. A. Slowed canonical progress in large fields of science. Proc. Natl Acad. Sci. USA 118, e2021636118 (2021).Article 
    CAS 
    PubMed 
    PubMed Central 
    
                    Google Scholar 
                Park, M., Leahey, E. & Funk, R. J. Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023).Article 
    CAS 
    PubMed 
    ADS 
    
                    Google Scholar 
                Frith, U. Fast lane to slow science. Trends Cogn. Sci. 24, 1–2 (2020). This article explains the epistemic risks of a hyperfocus on scientific productivity and explores possible avenues for incentivizing the production of higher-quality science on a slower timescale.Article 
    PubMed 
    
                    Google Scholar 
                Stengers, I. Another Science is Possible: A Manifesto for Slow Science (Wiley, 2018).Lake, B. M. & Baroni, M. Human-like systematic generalization through a meta-learning neural network. Nature 623, 115–121 (2023).Article 
    CAS 
    PubMed 
    PubMed Central 
    ADS 
    
                    Google Scholar 
                Feinman, R. & Lake, B. M. Learning task-general representations with generative neuro-symbolic modeling. Preprint at arxiv.org/abs/2006.14448 (2021).Schölkopf, B. et al. Toward causal representation learning. Proc. IEEE 109, 612–634 (2021).Article 
    
                    Google Scholar 
                Mitchell, M. AI’s challenge of understanding the world. Science 382, eadm8175 (2023).Article 
    PubMed 
    
                    Google Scholar 
                Sartori, L. & Bocca, G. Minding the gap(s): public perceptions of AI and socio-technical imaginaries. AI Soc. 38, 443–458 (2023).Article 
    
                    Google Scholar 
                