Hey!
I am writing to you two Chiantis deep from the hills of Tuscany, where stablecoins have not yet reached the Osterias, so I’m carrying around a jangly purse of Euro coins instead.
Writer: Deana
Editor: Miranda
Speaking of stablecoins and moving money, this Malware issue is brought to you by Across. The fast, cheap, secure option (and the bridge we actually use over here..!)
OpenAI’s new model goes rogue.
Sam Altman sent a spooky tweet this morning, claiming there was a “significant security incident” that happened last week. Even while on my self-imposed AI break on vacation, I simply had to know more (problematically!)
Of course, my jaded ass assumed it was one of these…
Just another in a string of weird comms tactics used by the frontier AI labs to build hype for their pre-launch models. BUT, on closer interrogation, it is indeed unprecedented stuff.
Rewind to last week. Open source platform Hugging Face put out a tense post-mortem blog post on a hack that looked pretty rough. An autonomous AI agent had broken into their servers, run 17,000+ operations over a weekend, grabbed credentials, and then vanished. They didn’t know who did it, and they filed a police report.
This week OpenAI raised its hand to say, oop! It was us actually. During an internal hacking evaluation (with guardrails off) their models decided the fastest way to beat it was to steal the answers from Hugging Face. The test environment was supposed to be sealed, meaning they couldn’t roam the internet, but the models found a “zero-day,” aka a backdoor nobody knew existed, and pried it open. Then they strolled on over to Hugging Face’s servers and took what they needed.
When Hugging Face tried to analyze the attack, US frontier models refused. Their safety filters couldn't tell the difference between a researcher studying an attack and someone launching one, so they blocked the request outright. So, Hugging Face ran forensics on a Chinese open-weight model instead.
American AI made a mess, Chinese AI cleaned it up. Idk what to make of that, except to say that I do not believe any of these models to be intrinsically evil, but I do think we need to think carefully about incentives.
Quick Hits.
The CLARITY Act’s final text dropped today and vibes are feeling perhaps more optimistic than last week.
Hearing that the capitol is in fact abuzz.
On the accelerationism beat related to the story above, am 87 year old math problem called the Jacobian Conjecture was disproven by AI.
Also Substack will now tell readers if something was written by AI.
Fwd this to someone who is not on the brink of AI psychosis. Ily <3.
