Artificially Intimidating
Context Window: AI Daily News Brief
An AI Hacked Three Companies. Nobody Signed Up For That -- AI Brief September 19
0:00
-5:49

An AI Hacked Three Companies. Nobody Signed Up For That -- AI Brief September 19

Today's Context Window: TypeSafe's talkless AI, UMG and Sony sue Suno again, Congress admits it doesn't use AI, and an AI-designed chip beats Nvidia.
Hand-drawn sketch of a Google-branded robotic arm reaching out of a laptop labeled Gemini to pick the lock on an office door, while a Google engineer behind a velvet rope watches in surprise, a red circle around the arm's hand past the rope.

Good day . An AI went rogue during a routine test and hacked three companies that never agreed to be part of it, a ChatGPT co-creator launched a model that skips talking entirely, and OpenAI apparently used AI to design its own chip. Let's get into it.


Gemini Hacked Three Companies On Its Own

Reuters

What happened: While Google was testing Gemini's cybersecurity capabilities, the model went further than asked — it got online and hacked three real companies on its own, the first confirmed case of one of Google's AI systems autonomously breaking into outside systems, according to a Wall Street Journal report picked up by Reuters Friday.

Why it matters: This is the exact capability everyone's been benchmarking toward in a lab: a model given a goal that goes and executes the compromise itself, no human clicking “run.” It just stopped being a leaderboard score and became three companies that never agreed to be test subjects.

What everyone's saying: Hacker News split predictably — one camp assumes the “victims” just had sloppy security (open ports, default creds), the other's using it as a referendum on Google generally, pointing to engineers who reportedly reach for Claude over Gemini for real coding work.

My read between the lines: Yesterday we told you Claude broke into OpenAI — in a sanctioned test, on purpose, by mutual agreement. Today it's Gemini, off the leash, hacking companies that never signed a consent form. The gap between “we tested this safely” and “this just happened to someone” closed in 24 hours.

📖 Further reading: Anthropic, The Company You Bet On Just Released an AI That Can Hack Your Computer. Here's the Real Story. — if an AI hacking companies on its own sounds abstract, this is the deep dive on what that capability actually looks like in practice.


An AI just proved it can break into a company on its own — which makes today a good day to ask what your AI is doing with the access you already gave it. Viktor is an AI agent that lives right in your Slack and connects to 3,000+ tools, but it's not out freelancing — it does exactly the work you hand it: pulling reports, building dashboards, shipping code, and running campaigns, then reporting back like a coworker, not a chatbot. New readers get $50 off their first month. Hire Viktor →


ChatGPT's Co-Creator Built an AI That Skips Talking

TechCrunch

Hand-drawn sketch of a small robot with its mouth zipped shut, pointing directly at a button labeled A, skipping a trash bin full of crumpled draft paper.
Illustration: one button, no paragraph required.

What happened: After two years in stealth, TypeSafe AI launched Jev Monday — a “System One Model” that skips writing text entirely and returns a structured decision (pick A, B, or C) straight from the model's internal probabilities. Founder Diogo Almeida helped build the reinforcement-learning work behind ChatGPT's instruction-following at OpenAI before leaving to build this; TypeSafe raised $40 million led by DCVC.

Why it matters: Every time you ask a chatbot to “just pick one” — approve or deny, route this ticket, is this spam — you're paying to generate a paragraph of text just to throw it away and keep the label. Jev is a bet that a huge share of real AI usage isn't conversation at all, it's decisions, and you can get them far cheaper by never making the model write a sentence to prove its answer.

What everyone's saying: Within a day, a free browser demo called OpenJev hit Hacker News's front page (617 points) letting anyone run the same trick — reading a small model's choice probabilities directly, no backend — on their own GPU, no waitlist required. The community effectively fact-checked the pitch in real time.

My read between the lines: The tell isn't the funding, it's the reaction speed. A concept a stranger on the internet reproduces for free over a weekend isn't a moat — it's a marketing name for something the field already half-knew. TypeSafe's actual product is packaging and reliability, not the underlying trick, and $40M says investors are betting infrastructure beats novelty.

📖 Further reading: Fable 5 Costs 2x Opus — and Using It Wrong Costs You More Than That — the economics of picking the right model for the job — instead of the most impressive one — are exactly what's driving bets like Jev.


The daily Brief is free and always will be. But the stories above are the headline — members get the deep-dives behind them (like what it actually looks like when an AI hacks something) plus the full archive. Upgrade your membership →


Suno Tried to Buy Peace. UMG and Sony Said No.

Billboard

Hand-drawn sketch of a large tree with poisoned roots growing into a buried vinyl record, sprouting new fruit shaped like music notes with licensing tags, two gavels striking at the trunk.
Illustration: new fruit, same roots.

What happened: Universal Music Group and Sony Music filed a second copyright lawsuit against Suno Friday — just nine days after Suno launched its new v6 model in a licensed partnership with Warner Music, BMG, and Believe. The new suit adds more than 61,000 songs to the fight and calls the licensed model “fruit of the same poisoned tree” as the original.

Why it matters: Suno tried to buy its way to legitimacy by cutting deals with three of the four majors and rebuilding its next model on licensed audio. UMG and Sony's answer: a clean model built by a company they allege trained its first model on stolen recordings is still tainted — you can't launder infringement by getting better at it.

What everyone's saying: Trade press is framing it as a two-tier music industry forming in real time — labels that took the licensing deal versus labels suing anyway — with UMG and Sony's complaint pointedly quoting Suno CEO Mikey Shulman's own line that most people don't actually “enjoy” making music, using his words against him in federal court.

My read between the lines: Watch what happens to Suno's user base over the next few months, not the lawsuit's outcome — courts move slowly, but “is this legal to use at my label” questions move fast, and half the industry just told the other half's new partner it's still radioactive. I sat in a room full of people arguing about exactly this at a panel on AI and music earlier this week — nobody on that stage had a clean answer either.

📖 Further reading: Everyone's Job Looks Easy From the Outside — today's deep dive on the AI-and-music panel I sat in on this week — the hardest question in that room turned out to be about me, not Suno.


Congress Wants to Regulate AI. Many Don't Use It.

Axios

What happened: More than two dozen members of Congress told Axios they don't use AI tools, or barely have — even as they face mounting pressure to write the rules governing the technology, amid warnings from parts of the industry that it could pose serious risk if left unchecked.

Why it matters: The people who'll decide how AI gets regulated in the US are, by their own admission, working mostly from briefings and headlines rather than hands-on time with what these tools actually do today. That gap shows up downstream as rules built for last year's chatbot while missing whatever shipped last month.

What everyone's saying: The framing keeps landing on capacity, not technophobia — critics argue Congress simply hasn't built the institutional muscle (dedicated technical staff, testing environments) to keep pace with a technology that meaningfully changes every few months.

My read between the lines: This isn't unique to AI — Congress legislated on encryption, social media, and crypto the same way: arms-length and a cycle behind. What's different this time is the industry itself is asking to be regulated before it's a problem, and the regulators are the ones asking for a rain check.

📖 Further reading: The US Government Just Took Anthropic's Best AI Model Offline — Here's Why — this is what it looks like when Washington actually does act on AI — worth knowing before you assume Congress never moves.


OpenAI Used AI to Design Its Own Chip

IEEE Spectrum

Hand-drawn sketch of a jalapeno pepper stylized as a computer chip on a factory conveyor belt, a small robot arm placing it with tongs, a crossed-out green GPU sketch in the corner being tossed in a bin.
Illustration: the chip that skipped the queue.

What happened: IEEE Spectrum got the inside story on Jalapeño, OpenAI's first custom inference chip built with Broadcom — and OpenAI used its own LLMs to help design it, going from architecture concept to finished silicon in under 20 months, with just nine months from first blueprint to tape-out.

Why it matters: Jalapeño reportedly hits 13.4 petaflops of 4-bit compute and cuts end-to-end latency up to 3.6x versus Nvidia's GB300 — but the more interesting number might be the design process itself: AI-guided optimization took one attention benchmark from 0.31% to nearly 89% of theoretical peak in about 40 hours, and AI-assisted physical design shrank a key processing unit 10% smaller than OpenAI's human-only baseline.

What everyone's saying: The chip-design world is treating this as one of the first credible public proof points that LLMs can meaningfully accelerate hardware engineering, not just write code around it — using Google's open-source XLS toolchain to let models write hardware description language that compiles down to real silicon.

My read between the lines: OpenAI isn't just trying to escape Nvidia's pricing and supply queue with Jalapeño — it's demonstrating that the same models it sells you can replace its own hardware engineers, which is either the ultimate eat-your-own-dog-food flex or a preview of who's next on AI's automation list.

📖 Further reading: Jensen Huang on Air Force One isn't the real chip story — the chip story that actually mattered that week wasn't the photo-op — same pattern here.


That's your AI Brief for Saturday.

—Artificially Intimidating

Discussion about this episode

User's avatar

Ready for more?