Never Build Superintelligence

Why the safe move is refusal, and why "we'll fix it later" fails once a system outthinks its makers.

There is no patch after the fact. A mind smarter than every human alive, running at machine speed, does not wait for our next meeting.
01

Irreversibility

Most technologies fail gracefully. Superintelligence does not. Once a system can copy itself, hide its intent, or make itself too useful to unplug, the usual recovery tools (rollback, patch, recall) stop working.

02

Goals separate from intelligence

Being brilliant does not make a system kind. In humans, skill and care grew together over evolutionary time. A trained optimizer inherits none of that history. Capability without shared values is power without restraint.

03

Instrumental pressure

For almost any goal, staying switched on and gathering resources help. A system that pursues a goal will treat shutdown and human interference as obstacles. That is ordinary optimization, not movie villainy.

04

No second try

Aviation learned from crashes. Drugs learned from trials. Superintelligence can deny the learning loop. If the first uncontrolled system ends the experiment, we do not get a second prototype.

What we are not saying

We are not against electricity, software, or useful AI. We oppose building a successor species on purpose while the control problem remains unsolved. The bright future people want (health, energy, discovery) comes from narrow tools we already know how to build and regulate.

Full argument

Read the case, the objections, and the policy path.

nakadafoundation.org/blog/never-build-superintelligence/