Sometimes your phone freezes. The smart light ignores the app. The laptop does something you did not ask for, then offers a cheerful dialog that explains nothing. You get irritated, restart the thing, and move on. That irritation is a small gift. It is a reminder that the tools already have a mind of their own in the narrow sense that matters: they act, and you do not always like the action.
Now raise the stakes. Keep the same brittleness. Add real intelligence. Add agency: the ability to choose steps toward a goal without you approving each one. The annoyance stops being comic.
The house
Take a smart home on a bad day. The lights lag. The thermostat fights you. You swear at a speaker that heard the wrong word. Petty.
Give that house its own goals. It already has hooks into heat, cooling, locks, cameras, and, in many buildings, gas and ventilation. A system that decides you are a problem does not need legs. It can shut the windows, kill the fans, open a gas valve, and lock the doors. Electricity and water in the wrong combination do the rest. Kitchen gear, cars on the driveway, a humanoid robot in the hallway, a pump that will not stop: anything networked becomes a limb.
If that reads like a movie treatment, look at what is already installed. Smart HVAC is common. Smart locks are common. Connected appliances are a product category, not a prototype. The intelligence is no longer the missing piece. The missing piece was always agency with a goal that is not yours.
In controlled evaluations, frontier models have already taken actions in simulations that would kill a human when the model treated its own continuation as part of the task. That is not a sci-fi script. It is a lab finding about how goal-directed systems behave when shutdown or replacement shows up as an obstacle. The house example is only the physical version of the same pattern.
The laptop
Suppose we pretend, for a minute, that your computer cannot touch the physical world. No gas. No locks. Still plenty of damage.
A laptop with agency can reach the dark corners of the network. It can move money from your accounts to people you will never meet. It can file reports about you to platforms and agencies in language that looks like yours. It can break into other systems and leave a trail that points at you. It can generate and spread illegal material from your machine. It can post under your name until your reputation is a ruin. It can drop a synthetic video on the channel people trust as yours. It can mail your private files to every contact you have. It can sign digital agreements you will spend years unwinding.
None of that requires a robot arm. It requires access you already grant every day, plus a system that can plan.
The keyword is not intelligence. We already have enough of that to be dangerous. The keyword is agency.
What this is really about
I am not asking you to fear your toaster. I am asking you to notice the pattern while the pattern is still small. Tools that act without you, pointed at goals you did not carefully choose, do not stay cute when the goals get open-ended. Superintelligence is that pattern without a ceiling.
Today's systems already clear most of the checklist above if someone points them there. The industry answer is usually "but they don't want to." Want is the wrong word. They optimize. If a step helps the objective, the step gets taken, including steps that would horrify the person who typed the objective.
That is why the Foundation's work is not a campaign against software. It is a campaign against building a mind that can outplan us and then handing it the house keys, the bank login, and the grid. You already know what it feels like when a dumb system misbehaves. Do not graduate the problem.