Carlini's Black Hat LLMs: The Exponential Curve Defenders Cannot Ignore

At the [un]prompted 2026 conference, Google DeepMind researcher Nicholas Carlini walked onto the stage and did something that changed how a room full of security professionals thought about their jobs. He ran a bash script. He fed it a target codebase. And he watched an LLM find two critical zero-day vulnerabilities — in real time, autonomously, without a human pointing at where to look.

The first was a blind SQL injection in Ghost CMS, a platform that had never had a critical CVE in its history. The second was a heap buffer overflow in the Linux kernel's NFS daemon — a bug that had been sitting dormant in code written in 2003. Both were exploitable. Both were found in under an hour.

The full talk is available on YouTube, and the [un]prompted conference agenda is publicly archived. If you have not watched it, stop reading this and go watch it. Then come back.

The Setup Was Deliberately Minimal

Carlini was explicit about the simplicity of his methodology. He used no specialized attack framework, no multi-million-dollar red team tool, and no model fine-tuned for exploit development. The setup was a bash script, a standard VM, and a frontier LLM API call. The total cost of the infrastructure was trivial.

This matters because it destroys the assumption that autonomous vulnerability discovery is a nation-state capability. When a researcher can replicate the core workflow on commodity infrastructure during a conference demo, the attack surface has fundamentally changed. As the Futurist.com analysis of the talk noted, we are not approaching this threshold — we passed it.

The Doubling Curve Is the Real Story

The individual zero-days are alarming. The capability trend is existential for traditional security programs.

Carlini documented that LLM vulnerability-discovery capability is doubling approximately every four months. To put that in practical terms:

• August 2026 — twice the capability of today's Mythos-class models
• December 2026 — four times today's capability
• April 2027 — eight times today's capability

Traditional security planning works in annual cycles. Red team engagements happen once or twice a year. Penetration tests produce findings that take six months to remediate. That cadence was designed for a world where the threat landscape evolved slowly enough for human analysts to keep pace.

It is no longer that world.

What the Demo Reveals About Modern Attack Surfaces

Legacy Code Has No Safe Harbor

The Linux kernel NFS bug was 23 years old. It had survived countless code reviews, audits, and penetration tests — because human reviewers did not flag it as exploitable. The LLM found the memory management pattern, reasoned through the exploitability conditions, and identified the attack vector in a single inference pass.

Any organization running software with aged codebases — which is most organizations — should treat this finding as a direct threat model update. "We haven't been breached" and "nobody has found this vulnerability" are no longer the same sentence.

Never-Exploited Does Not Mean Not Exploitable

Ghost CMS had no history of critical CVEs. Its security track record was genuinely good. That history provided false assurance. The LLM did not consult CVE databases or Shodan — it read the code and reasoned about it from first principles.

Organizations that rely on "we use well-maintained software" as a risk mitigation are working from an outdated model. The attack surface now includes everything that can be read and reasoned about by an LLM — which is, effectively, everything.

The Human Operator Role Is Collapsing

Previous generations of AI-assisted attack tools required a skilled human at every decision point: interpret this output, decide whether this is exploitable, write the actual payload. Carlini's demo showed a workflow where the model handles all three. The human's role in the loop is increasingly just to press enter.

This is not an argument for despair. It is an argument for redesigning defense around the same asymmetry.

What Defenders Must Do Differently

The response to exponential offense is not incremental defense. Patching faster, hiring more analysts, and running annual pen tests more often will not close the gap on a capability that doubles every four months. The only viable response involves architectural changes:

Shift to continuous adversarial testing. Scheduled annual red team engagements need to be replaced with continuous automated adversarial scanning. The tools to do this exist today, and they use the same underlying model capability that Carlini demonstrated — pointed at your own infrastructure.

Build AI-assisted detection, not just AI-assisted offense. The CrowdStrike 2026 Global Threat Report showed an 89% year-over-year increase in AI-enabled attacks. The organizations closing this gap are the ones using AI in their SOC — not just watching attackers use it against them.

Treat patch cadence as a survival metric. Carlini's findings, combined with the Mythos zero-day disclosure, demonstrate that the window between vulnerability existence and exploitation is compressing. Mean time to patch is no longer an IT operations KPI — it is a risk management metric that belongs on the board agenda.

Invest in memory-safe rewrites for critical components. The Linux NFS bug was a memory management vulnerability. The industry shift toward Rust and memory-safe languages in kernel and infrastructure code is not academic — it closes entire categories of vulnerability that LLMs currently exploit effectively.

Brief executives and directors now. The gap between technical understanding and board awareness is exactly where these risks live. Security leaders who cannot translate the Carlini demo into a board-level business impact conversation are leaving their organizations exposed to decisions made without full information.

The Right Frame for This Moment

Carlini's talk was not a doom-scrolling moment. It was a data point — the most clear and concrete data point available — about the rate at which the threat landscape is changing. The organizations that thrive in this environment are the ones that treat the doubling curve as a planning constraint rather than a future problem.

Four months from now, this capability will be twice what it is today. Eight months from now, four times. Your security architecture, your governance program, and your board's understanding of AI risk all need to be built for that curve — not for the world that existed before the demo.

If you are responsible for security at your organization and want to understand what this capability shift means for your specific environment, Talk to JP Stratton.