The future is a wild fantasy. It’s feverishly concocted out of our hopes, fears, biases, ignorance, and imagination, saying far more about us than what is to come. Predictions of the future are really just reflections of the present. And that means we’re really bad at predicting what the future will bring.

AI systems often only form a technical component that is embedded in a social process. Together, they make up a more comprehensive socio-technical system, and the impacts of a given AI system always depend on this larger context in which they are implemented. The technical specification of an AI system does not predetermine the impact that it will have and the risks that it entails when it is implemented. A given technological solution, such as a lipreading device, may well be harmless or even vastly beneficial in one setting, for example, for deaf persons, but cause profound ethical and regulatory problems as part of a public video surveillance system - meaning that when an AI application is transposed from one setting to a different setting it may have radically different social consequences.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.