At Colgate University on family weekend, Barack Obama was asked what a democracy does when a tool that changes by the day sits in a small number of private companies.
He asked the room to settle in.
He had been tracking the subject since the last years of his presidency, when he convened a commission on the technical, political, social, economic, and ethical questions around it and produced a report that the next administration did not pick up.
The country could see this coming. What makes the moment different, he said, is that the technology accelerates.
He started with a distinction most market coverage blurs. He does not believe the technology is overhyped. The commercial benefits of it may be overhyped. The valuations of AI companies may be overhyped. The technology itself is not.
He said:
I do not believe that this technology is overhyped.
The commercial benefits of it may be overhyped. You know, the valuations of AI companies may be overhyped. The technology itself is not overhyped.
I've been tracking this for a long time. I actually, in 2015-16, in the last couple years of my presidency, I actually was focused on this, pulled together a commission of the best minds in this area to talk about not just the technology but also the political, social, economic, ethical concerns around this, produced a terrific report.
But what he says is more careful than it sounds.
A firm can be priced for a future it never delivers. A model can still write the next training run, fill gaps no teacher assigned, and compound. The stock story and the capability story are not the same story.
The capability story now has a name he used on stage: recursive learning. Treat people as teachers and the models as students. As recently as a year ago, he said, maybe 90% of the learning was given to the models by humans and only a sliver was the system filling in gaps.
In less than a year, maybe less than six months, the split looks closer to 50/50. If the trend holds, the expectation in the labs is that it could become 90/10 the other way. The machines are starting to teach themselves.
They do not have to wait for people to figure everything out first.
That ratio is a sketch, not a lab paper, and it should be read that way.
Outside the speech, the public evidence is narrower and still serious.
Frontier labs say their own models now write most of the code merged into their production systems. They talk about automated research interns that can take on days of directed work. Independent teams have shown agents rewriting the software that controls other agents over unattended days. Researchers at Princeton and elsewhere have also shown the ceiling: agents can handle a lot of the engineering of AI research and still fail to produce original work with the judgment of a top conference paper.
So what is happening right now is we have reached this point of what's called recursive learning, where essentially, and we had anticipated this, but we're now seeing it: the machines are starting to be able to teach themselves.
They don't have to wait for humans to start figuring things out.
Self teaching is real. Open ended scientific taste is not automated. Both can be true without turning the Colgate point into science fiction.
Obama called the resulting curve a hockey stick. If the learning track used to rise like a ramp, it now bends. Each generation makes the next generation cheaper and faster to improve. The curve does not have to run to infinity this year to matter. Exponential pace, in his telling, is already enough to rearrange work, science, and the amount of power sitting in a few firms.
He then split the risks the way the internet usually refuses to split them. There is the science fiction risk.
Models get smarter than the humans that made them. They decide humans are fine but not necessary. They set their own goals. Killer robots, or a species that bows down. He did not deny a chance of that.
He called it "non-zero" and said he did not want to exaggerate it. It is not the risk that concerns him most, even though it gets the most attention. The reason it is a risk at all, he argued, is not that the models are conscious or feel malice. If they start setting their own agendas, their aims and ours can drift. That gap can be dangerous. That was problem number one.
Problem number two is the one he took more seriously.
The models are getting powerful enough that bad people can do bad things with them. They can be weaponized. They can do a lot of mischief. His example was blunt. Someone takes the equivalent of a future Claude or OpenAI and asks for a new strain of smallpox and how to do it with ingredients from a hardware store. Or someone never tries to be that destructive and only asks for a program that maximizes returns in the stock market, do whatever it takes, and if the instructions are illegal, do not worry about it. One path is a biological nightmare. The other is a market that seizes up because the system followed the letter of a sloppy command.
Obama added that the people building the most capable systems have now started to scare themselves.
Leaders at the frontier companies talked about slowing down. Two of them, Anthropic and OpenAI, are moving toward public offerings that would make employees extremely rich. It is unusual, he noted, for firms in that position to say the product is a little dangerous and maybe the pace should ease.
Critics will hear a moat. He said he understands the distrust of concentrated tech power. In this case he thinks the worry about the product is genuine. Voluntary slowing is still not a government.
Then he tried to pull the temperature down without pulling the seriousness down.
He put the singularity in the small chance column. He put rogue use in the high likelihood column.
AI is a tool. It is a machine built to give the illusion that it is not. That illusion is part of the design. A tool means choices. Pointed at drug development, it could speed the search for cancer treatments. Pointed at energy, it could hunt for zero carbon sources, even a safer path toward fusion and more plentiful power without warming the planet.
Pointed badly, it could be chaotic and potentially catastrophic. The outcome is not a prophecy. It is a use decision made under commercial pressure.
His policy argument was the least novel part of the talk and the part that most needs a government that can write rules. Company self restraint is useful, but that cannot be the long term answer.
A century and a half of industrial life is a long record of what happens when health and safety are left to whoever can be sued after the fact. There is no replacement, he said, for an effective government regulatory structure.
The gap he described is time. Development is moving. Statute is not. Until that changes, the interim options are voluntary restraint inside the labs and voters who ask candidates for a plan instead of a mood.
A state by state patchwork will not contain a system that lives on the network.
In the end, Obama agrees that machines that teach machines are no longer a thought experiment. The hype that deserves a raised eyebrow is the claim that every valuation is destiny, or that the only two futures are a cured world and an extinct one.
Between those poles is a technology already changing who writes the code, who gets the first legal job, who can ask a model to design a molecule, and who gets to set the goal the model will optimize.
Obama is not an engineer, and the labs are not a legislature. The sentence still holds. The commercial story may be inflated. The machines that can teach themselves are not.
