The more artificial intelligence enters our lives, the more important become ethics and philosophy. Below are four book recommendations.
The book begins by positing a scenario in which AI has exceeded human intelligence and become pervasive in society. Tegmark refers to different stages of human life since its inception: Life 1.0 referring to biological origins, Life 2.0 referring to cultural developments in humanity, and Life 3.0 referring to the technological age of humans. The book focuses on "Life 3.0", and on emerging technology such as artificial general intelligence that may someday, in addition to being able to learn, be able to also redesign its own hardware and internal structure.
James Barrat weaves together explanations of AI concepts, AI history, and interviews with prominent AI researchers including Eliezer Yudkowsky and Ray Kurzweil. The book starts with an account of how an artificial general intelligence could become an artificial super-intelligence through recursive self-improvement. In subsequent chapters, the book covers the history of AI, including an account of the work done by I. J. Good, up to the work and ideas of researchers in the field today.
Throughout the book, Barrat takes a cautionary tone, focusing on the threats artificial super-intelligence poses to human existence. Barrat emphasizes how difficult it would be to control or even to predict the actions of, something that may become orders of magnitude more intelligent than the most intelligent humans.
Superintelligence: Paths, Dangers, Strategies is a 2014 book by the Swedish philosopher Nick Bostrom from the University of Oxford. It argues that if machine brains surpass human brains in general intelligence, then this new superintelligence could replace humans as the dominant lifeform on Earth. Sufficiently intelligent machines could improve their own capabilities faster than human computer scientists, and the outcome could be an existential catastrophe for humans.
It is unknown whether human-level artificial intelligence will arrive in a matter of years, later this century, or not until future centuries. Regardless of the initial timescale, once human-level machine intelligence is developed, a "superintelligent" system that "greatly exceeds the cognitive performance of humans in virtually all domains of interest" would follow surprisingly quickly, possibly even instantaneously. Such a superintelligence would be difficult to control or restrain.