superintelligent agent that will aid its creators, while avoiding inadvertently building a superintelligence that will harm its creators. Some scholars argue that solutions to the control problem might also find applications in existing non-superintelligent AI. Major approaches to the control problem include alignment, which aims to align AI goal systems with human values, and capability control, which aims to reduce an AI system's capacity to harm humans or gain control. An example of "capability control" is to research whether a superintelligence AI could be successfully confined in an "AI box". According to Bostrom, such capability control proposals are not reliable or sufficient to solve the control problem in the long term, but may potentially act as valuable supplements to alignment efforts. Warnings Physicist Stephen Hawking, Microsoft founder Bill Gates, and SpaceX founder Elon Musk have expressed concerns about the possibility that AI could develop to the point that humans could not control it, with Hawking theorizing that this could "spell the end of the human race". Stephen Hawking said in 2014 that "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." Hawking believed that in the coming decades, AI could offer "incalculable benefits and risks" such as "technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand." In January 2015, Nick Bostrom joined Stephen Hawking, Max Tegmark, Elon Musk, Lord Martin Rees, Jaan Tallinn, and numerous AI researchers in signing the Future of Life Institute's open letter speaking to the potential risks and benefits associated with artificial intelligence. The signatories "believe that research on how to make AI systems robust and beneficial is both important and timely, and that there are concrete research directions that can be pursued today." Prevention through AI alignment See also Philosophy of artificial intelligence Artificial intelligence arms race Autonomous robot Industrial robot Mobile robot Self-replicating machine Cyberocracy Effective altruism Existential risk from artificial general intelligence Future of Humanity Institute Global catastrophic risk (existential risk) Government by algorithm Human extinction Machine ethics Machine learning/Deep learning Transhumanism Self-replication Technophobia Technological singularity Intelligence explosion Superintelligence Superintelligence: Paths, Dangers, Strategies Notes References External links Automation, not domination: How robots will take over our world (a positive outlook of robot and AI integration into society) Machine Intelligence Research Institute: official MIRI (formerly Singularity Institute for