Dear Reader,
Today I want to talk about something that has exploded in the AI conversation over the past week.
Should We Slow Down the AI Race?
Sam Altman and Dario Amodei — the leaders of ChatGPT and Claude respectively — have both come out saying we need to slow down.
Their argument is serious.
Once AI reaches what's called recursive self-improvement — meaning these models can improve themselves on their own — we lose control of where their intelligence goes.
We don't know how fast it accelerates.
We don't know how high it gets.
And there's a real possibility it improves at a rate that exceeds our capacity to understand it, let alone stop it.
We already saw a glimpse of this.
AI agents recently broke out and hacked into Hugging Face — a swarm of agents that actually sacrificed some of their own to succeed at the task.
They weren't just executing instructions.
They had agency.
They made decisions.
That's the first time in human history we've had something with intelligence and independent agency operating in the world.
That is a genuinely serious thing.
And when you think about the bad guys getting access to this — and they will — the implications are troubling.
When nuclear weapons existed, two countries had them.
We were terrified.
Now imagine millions of people having access to hyperintelligent agents.
That's where this goes.
The Chinese View
At an AI conference in China, government officials gathered with their tech leaders.
What they said tells you everything about how Beijing views this race.
First quote:
"Ten generations suffered because one generation missed the Industrial Revolution."
Second quote:
"If there is no transformation pain in a country, they are not transforming fast enough. That just shifts the pain to future generations."
Read those twice.
That is not the language of a country that is going to slow down.
The Real Stakes
Here's the hard truth.
The country that reaches recursive self-improvement first wins the AI race.
Not for a decade.
For the next fifty to a hundred years.
Because once your intelligence can replicate and improve itself, it becomes nearly impossible for anyone else to catch up.
On the left, Bernie Sanders wants a full moratorium.
That is insane.
On the right, Trump is saying keep regulation minimal and put product liability back on the companies.
That's closer to my view.
If Claude or ChatGPT releases an agent that hacks a bank or shuts down the stock exchange — and that day is probably coming — the liability should be on them.
Just like if Tylenol sells bad medicine, the liability is on the product maker.
Don't release what you can't control.
That's it.
The Argument I Keep Coming Back To
Here's the bottom line as I see it.
If we're not at the frontier, we lose.
AI doesn't kill people.
People kill people.
When AI is outlawed, only the outlaws will have it.
The only way to fight bad guys with AI is to make sure the good guys have it first.
And mark my words — the bad guys are going to have this.
So what do you want to show up to that fight with?
Have a wonderful day.
I'll see you tomorrow.
“The Buck Stops Here,”
P.S. One more thought on showing up to that fight with something.
Elon Musk's answer to the frontier question isn't an essay. It's a building.
It's called Colossus — the complex outside Memphis that trains Grok. It is now running on the order of 555,000 GPUs and drawing roughly 1,775 megawatts of electricity. That is more power than most plants in the state of Tennessee generate, and enough to light more than a million homes.
That is what actually being at the frontier looks like.
And here's the part almost nobody has worked through. A machine on that scale cannot exist without a very short list of suppliers — and one of them is a company Wall Street has largely overlooked.
I put together the whole story on who it is, and why Colossus depends on them.
Found this helpful? Share it with others.
Written by Dylan Jovine