AI Security Scanners Are Calling In Our Technical Debt

A bug report landed in the PHP project's tracker back in 2024. It described an out-of-memory condition — the kind of thing that makes a process fall over, which is annoying but, in the informal hierarchy of bugs, basically a shrug. It sat there. Then, during an independent audit of PHP sponsored by OpenAI, a researcher at the security firm Calif — working with AI security scanners and his own deep expertise — spent roughly two weeks turning that nuisance crash into remote code execution. Any server running a vulnerable PHP build could be coerced into running an attacker's code or coughing up sensitive data. The sites in the blast radius included NASA, Wikipedia and the White House, because WordPress and Drupal run on PHP, and PHP runs on a startling fraction of the web.

The patch shipped before anyone malicious got there, so the story ended well. But the researchers involved keep saying the same thing: this is the beginning, not the end. Four decades of human-written code contains an ocean of bugs we collectively classified as 'crashes, won't fix,' and AI tooling has collapsed the cost of finding out which of those crashes are actually loaded weapons. I've spent the past few weeks talking to people who run security for real systems about what this changes in practice. What follows isn't a survey — it's a set of mistakes I've watched teams make already, and what the people who got it right did instead.

Mistake One: Filing Crash Bugs as 'Unpleasant but Harmless'

The PHP bug at the heart of this story wasn't hidden. It was reported, acknowledged and deprioritised — because it 'only' caused an out-of-memory error. This is the single most common triage failure in mature software organisations, and AI-assisted exploit development has just made it expensive. Columbia professor Suman Jana, who has worked on exactly this class of research, put the dynamic plainly in interviews around the disclosure: at any given moment every big project has a pile of crash reports nobody takes seriously, because converting a crash into an exploit requires long chains of tedious, hyper-specific reasoning. Humans avoid that work. Models don't get bored.

Here's the mechanical version of why the distinction matters. A memory bug that corrupts the heap with uncontrolled data usually just kills the process — the allocator notices something inconsistent and aborts, or the process dereferences garbage and segfaults. But the same bug, fed carefully sized and shaped input, becomes a write primitive:

void handle_request(const char *input, size_t len) {
char *buf = malloc(64);
memcpy(buf, input, len);   /* len never validated */
/* len = 65:            heap metadata trashed -> likely abort, filed as 'crash' */
/* len = attacker-tuned: adjacent object's function pointer overwritten */
process(buf);             /* calls through the corrupted pointer -> their code runs */
free(buf);
}

The line between those two outcomes isn't the bug — it's the input. Working out which bits to overwrite, in what order, to land execution at a chosen address is painstaking but mechanical, and 'painstaking but mechanical' is the exact centre of what language models are good at assisting with. The fix for this mistake is conceptual before it's technical: stop using 'it only crashes' as a severity signal for memory-unsafe code. A crash in C or C++ (or in the C core of an interpreter like PHP) is an exploit that hasn't been developed yet. The new triage question isn't 'does it crash?' but 'is the corruption attacker-influenced?' If untrusted input controls the size, the offset or the contents of a bad write, assume exploitability and prioritise accordingly — even when nobody has demonstrated a working exploit. Especially then, because demonstrated exploits now take weeks, not years.

Mistake Two: Treating Scanner Output Like a Backlog

The second failure mode appears the moment a team actually runs modern AI-assisted scanners across an estate that includes twenty-year-old code. The tool comes back with hundreds of findings, the dashboard turns red, and the team does what backlogs train them to do: sort by severity score and start at the top. Six months later they've fixed forty things, the count has gone up, and morale is wrecked. The engineers I spoke to who handle this well all said some version of the same thing: the output of an AI scanner is not a backlog, it's raw intelligence, and it needs an analyst's triage pass before a single ticket is filed.

Effective triage in this regime asks exploit-path questions, not severity-score questions:

• Is the vulnerable code reachable from untrusted input? A heap overflow in a function that only ever processes a local config file is a different animal from the same overflow in request handling.
• Is it internet-facing today? Not 'should it be' — is it. Check load balancer configs and DNS, not the architecture diagram.
• Does input control the corruption? Size, offset or contents of the bad access. This is the crash-vs-exploit line from the previous section.
• What does an exploit buy? A crash in a sandboxed worker that respawns is different from one in the process holding database credentials.
• Is there an exploit in the wild or a public write-up? That jumps anything to the front of the queue regardless of score.

That last point deserves emphasis. Scores like CVSS were designed for a world where humans found bugs at human rates, and they systematically overweight theoretical severity against reachability. When a scanner can generate theoretical severity at industrial scale, you drown. The teams coping best have inverted the pipeline: reachability and exposure first, severity second, and anything unreachable-by-construction gets documented and closed rather than fixed — because a fix you ship carries its own regression risk, and you'll need that risk budget for the findings that matter. This is also where AI code comprehension checks earn their keep on the defensive side: the same tooling that finds the bug can often answer 'show me every call path from the network boundary to this function' in minutes, a question that used to take a senior engineer a day per finding.

Mistake Three: Assuming Patching Can Move at Finding Speed

Here's an asymmetry that should be printed on a poster in every security team room. Finding the PHP bug's exploit path took one skilled researcher with AI assistance about two weeks. Developing the patch took four engineers two months. Finding is a search problem: you need one path through. Fixing is a proof-shaped problem: you need every path closed, and you need the billion installs that aren't being attacked to keep working exactly as before. The defender's job is strictly harder than the attacker's, and AI compresses the attacker's side of that gap far faster than the defender's.

Teams that ignore the asymmetry make one of two bad bets. Either they rush patches to match disclosure pressure and ship regressions — the PHP patch touched interpreter internals used by roughly three-quarters of websites that publish their stack, so the blast radius of a bad fix is civilisational — or they despair and do nothing while waiting for a perfect fix. The practitioners who handled this window well did something less heroic and more effective: they separated 'mitigate now' from 'fix properly'. Mitigations are cheap, fast and reversible — restrict the affected endpoint at the reverse proxy, deploy a WAF rule that blocks the exploit's input shape, drop the vulnerable code path behind a flag, shrink who can reach the service while the real patch matures. I've written before about how the patch-vs-mitigate decision is a genuine trade-off rather than a moral failing, and this incident is that lesson at planetary scale.

The corollary nobody wants to hear: your mean-time-to-mitigate is now the metric that matters, because the window between 'bug found by white hats' and 'bug found by someone worse' is shrinking. In the PHP case the window stayed open only because the researchers were responsible and the foundation moved fast. The Brotli compression library told a less flattering story in the same period — a comparable bug sat in the code for eight years, was fixed in git but left unreleased for eight months, and was independently rediscovered by at least four researchers. When discovery is cheap and parallel, 'someone will find it eventually' stops being a comforting thought and becomes a scheduling guarantee.

Mistake Four: Trusting the Fix Because an LLM Wrote It

The optimistic counterargument to all of this goes: fine, AI finds bugs faster than we can fix them today, but AI will fix them too. Anthropic released tooling in October along exactly these lines — an LLM generates a patch, then a second model generates a machine-checkable proof that the patch preserves the intended behaviour, and a small, dumb, trustworthy proof checker validates the proof. The checker is the only component that has to be correct, and it can be, because it does almost nothing. Jana is genuinely enthusiastic about this direction, and I understand why: it's the first credible path to making fixing nearly as automatable as finding.

But I've now watched two teams adopt 'AI-generated security patch' workflows and both made the same mistake: they treated the model's patch as a solution rather than a candidate. In one case the patch closed the reported heap overflow by adding a length check — and moved the overflow to a second allocation path six lines away that the model hadn't modelled. The fix passed review because it looked obviously correct, which is exactly the failure mode: LLM patches are fluent in a way human patches aren't, and fluency suppresses scrutiny. The teams that got value from generated patches added three gates, in this order:

1. A regression suite that includes the exploit input. The proof-of-concept that triggered the finding becomes a permanent test. If the scanner gave you a crashing input, that's your first new test case, free of charge.
2. A variant hunt. Before merging, someone (or something) answers: where else does this same pattern occur? Fixing one instance of a systemic pattern is how you get CVE sequels.
3. Independent verification of the invariant, not the diff. State what must now be true — 'no path from request handling reaches memcpy with unvalidated length' — and check that, with tests, static analysis or formal tooling. Reviewing the diff alone just re-reads the model's homework.

Formal verification will help here as it matures, but note the division of labour that makes it trustworthy: the creative parts are allowed to hallucinate, because the final arbiter is mechanical and small. That's the design principle to steal regardless of your tooling. Anything an LLM produces — finding, patch or proof — should terminate in something that doesn't share its failure modes. If your verification step is another prompt, you haven't verified anything.

Mistake Five: Assuming Someone Else Audited the Code

Underneath every technical lesson here sits an economic one. PHP is infrastructure on the scale of a public utility — it powers WordPress, Drupal and a long tail of the web that includes space agencies and seat-of-government websites — and its security rests on the PHP Foundation, a small funded body, plus volunteers. The comfortable myth of open source was always 'given enough eyeballs, all bugs are shallow.' What actually happened for thirty years was that everyone assumed the eyeballs belonged to someone else. AI is now supplying the eyeballs, indiscriminately, to attackers and defenders alike — which means the myth is finally being made true, just adversarially and on a deadline.

The PHP2Shell disclosure worked because of an unusual arrangement: OpenAI sponsored an external audit, a well-resourced firm did the work, and Wiz helped confirm and expand the findings. That's charity, not a system, and it doesn't scale to the thousands of critical-but-unfunded projects in your dependency tree. If your organisation's production depends on an open-source interpreter, framework or library — and it does — the fixes that matter are boring ones: fund the foundations, sponsor audits, contribute engineer-hours, and when you run your own scanners and find something in a dependency, report it upstream instead of patching your fork and moving on. The alternative is the world Johns Hopkins' Matthew Green predicts: nothing structural changes until an incident big enough to upset normal people forces it. His advice in the meantime is unglamorous and correct — when the update prompt appears, install it now, not next week. We learned the same lesson the hard way in the last big PHP disclosure cycle, when pulling the plug beat clever fixes for the teams that couldn't patch fast enough.

The eyeballs arrived. They're just not picky about which side they're on.

The Monday-Morning Playbook for AI-Surfaced Vulnerabilities

Everything above compresses into a short list of things I'd actually do this quarter if I ran an engineering org with any legacy code in production — and if you're reading this, you have legacy code in production:

• Patch aggressively and measure how long it takes. Your update latency is now a security control with a shrinking safe threshold. Know your number before you need it.
• Re-triage your existing crash-bug backlog with the exploitability question, not the severity score: does untrusted input control the corruption? Anything memory-unsafe and reachable gets promoted.
• Run AI-assisted scanning yourself, on your oldest code first. The marginal cost of rediscovery is now near zero, which means you should assume anything findable in your attack surface will be found. Find it first.
• Separate mitigation from fixing in your incident process. Pre-build the playbook: proxy rules, feature flags, access restriction. Buying time is a legitimate strategy; pretending you don't need time isn't.
• Gate generated patches on exploit-input tests and variant hunts, never on how clean the diff looks.
• Put money or hours into the upstream projects you depend on. Sponsored audits found this one. That's not a coincidence; it's the model working, and it needs to stop being exceptional.

There's a fair debate about whether this era is 'paying off technical debt' or signing us up for a treadmill of feature-neutral fix-work forever. Probably both: each non-trivial patch carries the risk of the next subtle bug, and no system is ever finished. But I'd rather live in the world where the debt is being called in by auditors and white hats than the one where we keep pretending the quicksand is bedrock because nothing has swallowed us yet. PHP2Shell ended with a patch and a sigh of relief. The next one is already in somebody's crash queue, filed as harmless. Go look at yours.