A Model of Pathways to Artificial Superintelligence Catastrophe for Risk
  and Decision Analysis
  An artificial superintelligence (ASI) is artificial intelligence that is
significantly more intelligent than humans in all respects. While ASI does not
currently exist, some scholars propose that it could be created sometime in the
future, and furthermore that its creation could cause a severe global
catastrophe, possibly even resulting in human extinction. Given the high
stakes, it is important to analyze ASI risk and factor the risk into decisions
related to ASI research and development. This paper presents a graphical model
of major pathways to ASI catastrophe, focusing on ASI created via recursive
self-improvement. The model uses the established risk and decision analysis
modeling paradigms of fault trees and influence diagrams in order to depict
combinations of events and conditions that could lead to AI catastrophe, as
well as intervention options that could decrease risks. The events and
conditions include select aspects of the ASI itself as well as the human
process of ASI research, development, and management. Model structure is
derived from published literature on ASI risk. The model offers a foundation
for rigorous quantitative evaluation and decision making on the long-term risk
of ASI catastrophe.
