27 Oct 2026
Securing AI Coding Assistants: Why Copilot-Generated Code Still Needs a Human Security Review
AI coding assistants write functional code fast, but functional isn't secure. What the research shows about AI-generated vulnerabilities.
AI coding assistants have gone from a novelty to a default part of how a lot of code gets written in 2026. They’re fast, they’re often right, and they’ve genuinely changed how quickly a small team can ship. What they haven’t changed is the fact that code that works and code that’s secure are two different bars, and AI-generated code clears the first one far more reliably than the second.
This isn’t a reason to ban AI coding tools. It’s a reason to treat their output the way you’d treat code from any fast, confident contributor who hasn’t necessarily been trained on your security requirements: useful, often correct, and not something that goes straight to production without a review step.
What the research actually shows
The widely-cited “Asleep at the Keyboard?” study (Perry et al., Stanford) ran an AI coding assistant across 89 scenarios covering common vulnerability classes and found that of the programs it produced, a substantial share — roughly 40% — contained exploitable vulnerabilities. That study is a few years old now and AI coding tools have improved since, but the underlying mechanism it identified hasn’t gone away: these tools are trained on a vast corpus of public code, and public code includes a lot of exploitable patterns. A model trained on that corpus will, with some frequency, reproduce them.
More relevant for where things stand in 2026: a peer-reviewed evaluation presented at the 39th Canadian Conference on AI (Amro & Alalfi) tested whether Copilot’s own built-in code-review feature could catch known vulnerabilities in labeled vulnerable open-source code. It found that Copilot’s code review frequently failed to flag critical vulnerability classes, including SQL injection, cross-site scripting, and insecure deserialization — exactly the kind of high-severity, well-understood bugs a mature SAST tool or an experienced reviewer would be expected to catch.
The practical takeaway from both studies together: AI tools are good at producing code that looks right and AI review tools are inconsistent at catching when that code isn’t right. Relying on the same family of tool to both write and self-review your code removes an important layer of independent verification.
Why this happens, not just that it happens
It helps to understand the mechanism, because it tells you where to focus review effort:
- Training data includes insecure patterns. Public repositories contain a huge amount of functional-but-flawed code — tutorials, abandoned projects, code written before a vulnerability class was well understood. A model trained on that corpus has seen plenty of examples of the insecure pattern, sometimes more often than the secure one.
- The model optimizes for “looks like working code,” not “is secure code.” Nothing in how these tools are typically used forces a security-correctness check; the signal they’re optimizing toward is closer to “plausible and syntactically correct” than “hardened against the specific threats relevant to this code path.”
- Context gets lost across suggestions. An assistant suggesting the next few lines of a function often doesn’t have full visibility into how that function’s output is used three files away — which is exactly the kind of cross-file context that input-validation and injection bugs depend on.
- Confidence doesn’t correlate with correctness. AI-generated code and AI-generated code review comments are both delivered with the same confident tone whether they’re right or wrong, which makes it easy for a reviewer to under-scrutinize a suggestion that reads as polished and complete.
Where to focus review effort
Not every line of AI-suggested code carries equal risk. Prioritize review on the areas where the research shows the most consistent gaps:
- Input handling and validation. Anywhere user-controlled data enters the system — form fields, API parameters, file uploads, query strings — review for injection risk (SQL, command, template) rather than assuming the assistant handled sanitization.
- Authentication and session logic. AI suggestions in this area tend to be functionally plausible (the login “works”) while missing edge cases like session fixation, improper token expiry, or weak password-reset flows.
- Deserialization and file parsing. Insecure deserialization was specifically flagged as a class Copilot’s own review missed — treat any
pickle-style, XML, or custom binary deserialization touching external input as high-risk by default. - Dependency and package suggestions. AI assistants will confidently suggest
importstatements or package names, occasionally for packages that don’t exist or aren’t the one you intended — a pattern increasingly exploited through typosquatting. Verify any newly suggested dependency against the real package registry before accepting it. - Error handling and logging. Watch for AI-suggested code that logs sensitive data (tokens, passwords, full request bodies) for debugging convenience, or that returns overly detailed error messages to the client.
- Secrets and configuration. AI assistants will sometimes suggest a hardcoded placeholder credential or API key “for now” — exactly the pattern that ends up committed and forgotten. (See our secrets management guide for how to avoid that entirely.)
A practical review workflow
You don’t need a new process — you need your existing one applied consistently to AI-suggested code:
- Treat AI suggestions as a first draft, not a final answer, especially in the six risk areas above. A fast “this looks right” skim is not the same as a review.
- Keep SAST tooling in the pipeline regardless of who (or what) wrote the code. Static analysis doesn’t care whether a human or a model produced the pattern it’s flagging — run it on every commit, AI-assisted or not.
- Don’t let AI-generated review comments substitute for human or automated review. If your workflow uses an AI code-review feature, treat its sign-off as one more data point, not a gate — per the research above, it has specific, documented blind spots.
- Require the same scrutiny for “small” AI-suggested changes. A one-line AI suggestion to “fix” a function is still a code change touching production logic; review it with the same rigor as a one-line human-written change in the same risk area.
- Train the team on what to look for, not just how to prompt the tool. Most engineers have ramped up fast on using AI coding assistants productively; fewer have had explicit training on where that output tends to go wrong.
Key takeaways
AI coding assistants reliably produce code that works; the research — from the original “Asleep at the Keyboard?” findings through a 2026 peer-reviewed evaluation of Copilot’s own review feature — consistently shows that “works” and “secure” aren’t the same bar, and that relying on AI tooling to both write and self-review code removes an important independent check. The fix isn’t abandoning these tools; it’s applying your existing review discipline (human review plus SAST) to their output with the same rigor you’d apply to any other contributor’s code, with extra attention on input handling, auth, deserialization, dependencies, and secrets.
Shahid Security’s Custom Software Development service builds security review into the development process from the start, whether your team is writing code by hand, with AI assistance, or both.
Want a security review process that accounts for how your team actually builds software today? Book a free consultation.
Sources: Amro, M. & Alalfi, M. H., “GitHub’s Copilot Code Review: Can AI Spot Security Flaws Before You Commit?”, 39th Canadian Conference on AI (2026); Perry, N. et al., “Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions” (Stanford).
FAQ
Does this mean we should stop using AI coding assistants?
No — the research doesn't say AI coding assistants are unusable, it says their output needs the same scrutiny as code from a fast, inexperienced contributor: useful, often correct, but not something you merge without review. Most teams get real productivity gains from AI-assisted coding; the fix is adding a review step, not removing the tool.
Can AI code review tools catch what AI code generation gets wrong?
Not reliably on their own. A 2026 peer-reviewed evaluation found that Copilot's own code-review feature frequently missed critical vulnerability classes like SQL injection, XSS, and insecure deserialization in labeled vulnerable code. Treat AI-generated review comments as one input, not a substitute for a human reviewer or a dedicated SAST tool.
What's the single highest-impact change we can make?
Require that any AI-generated code touching authentication, input handling, file access, or external data goes through the same human and automated review as hand-written code in those areas — no exceptions for 'the AI wrote it, it's probably fine.' That one policy change catches most of what the research flags.

Written by Shahid Iqbal
Founder & Cybersecurity Consultant, Shahid Security
