I Built an AI Code Reviewer. Then OWASP Broke It.

Most AI coding tools are very good at helping developers write code.

But I wanted to explore a different question:

Can AI help developers find the edge cases and security problems they didn't think about?

That question led me to build EdgeGuard, an open-source VS Code extension designed to investigate code from an adversarial perspective.

The idea is simple:

Try to break the code before production does.

But when I first tested the idea against real security benchmarks, I discovered a problem.

The AI was too optimistic.

And OWASP exposed it very quickly.

The False Negative Trap

I started testing EdgeGuard against the OWASP Benchmark for Java.

One of the first things I noticed was surprisingly simple.

Given code like this:

String param = request.getParameter("id");
String bar = DatabaseHelper.doSomething(param);
String sql = "SELECT * FROM USERS WHERE ID='" + bar + "'";

The LLM could see the untrusted HTTP input.

It could see the SQL construction.

But it couldn't see what doSomething() actually did.

So it made an optimistic assumption:

"This is probably an internal helper that sanitizes the input."

And the result could be:

SAFE

even though the data was still flowing into a SQL sink.

This was exactly the kind of false negative I wanted EdgeGuard to find.

The problem wasn't that the model couldn't understand SQL injection.

The problem was that the model was filling in missing information with an optimistic assumption.

Stop Guessing About Unknown Code

I changed the investigation strategy.

Instead of sending raw code and asking:

"Is this vulnerable?"

EdgeGuard now provides explicit security assumptions and asks the model to preserve taint unless there is evidence that it has been sanitized.

One important rule became:

If tainted data enters an unknown function, treat the data as still tainted unless there is evidence that the function sanitizes it.

The investigation also became evidence-oriented instead of relying on a simple SAFE or VULNERABLE classification.

For example:

• Step 1: Untrusted input is received from an HTTP request.
• Step 2: The input enters an unknown helper, so the taint state is preserved.
• Step 3: The tainted value is concatenated into a SQL query.
• Conclusion: The data flow represents a potential SQL injection.

That small change made a big difference.

The AI was no longer being asked to guess what an unknown function probably did.

It had to reason from the available evidence and follow explicit security assumptions.

The Scale Nightmare

Finding a potential vulnerability in one function is one thing.

Doing it across thousands of functions is another.

Large projects contain enormous numbers of methods that simply aren't interesting from a security perspective:

get()
set()
ToString()
Equals()
simple CRUD methods
data mapping
utility functions

Sending all of them to an LLM would be wasteful.

It would also create two immediate problems:

API rate limits and API cost.

My philosophy has always been:

Start with the simplest solution, and only add complexity when the simple thing breaks.

So I didn't want the LLM to analyze everything.

I added a local Static Risk Screening stage that runs directly inside VS Code.

Before making an API call, EdgeGuard parses the code locally and looks for characteristics that make a function worth investigating.

For example:

• database sinks
• file-system access
• process execution
• network boundaries
• user-controlled input
• potentially dangerous APIs
• missing validation

The local stage acts as a filter.

Instead of:

7,000 functions
        ↓
     LLM
        ↓
  $$$$$$$$$

the architecture becomes:

7,000 functions
        ↓
Local Static Screening
        ↓
High / Medium Risk
        ↓
LLM Investigation
        ↓
Evidence + Verification

This makes the LLM a reasoning engine, rather than the first line of analysis.

The Stress Test: 7,536 Functions

I wanted to know whether this architecture would actually work at project scale.

So I loaded the entire OWASP Benchmark Java project into VS Code and triggered a workspace scan.

The scan discovered:

• 7,536 functions
• 2,771 files

The local screening stage first filtered the code and identified functions that deserved deeper investigation.

The LLM then focused on the higher-risk candidates instead of blindly processing every function.

In this test, EdgeGuard reported:

2,145 potential defects

The findings included security issues such as:

• SQL injection
• command injection
• unsafe data flows
• other potentially dangerous input-to-sink paths

The important part for me wasn't simply the number of findings.

It was that the architecture could move from:

one function → one LLM request

to:

whole workspace → local screening → targeted AI investigation

without turning every method into an API call.

Testing Three Languages

I also wanted EdgeGuard to work beyond Java.

So I tested the architecture against projects in three different ecosystems:

Java

OWASP Benchmark

TypeScript

OWASP Juice Shop

C

Microsoft eShopOnWeb

This introduced another problem.

Java, C#, and TypeScript have very different:

• syntax
• AST structures
• language conventions
• security APIs
• project structures

I didn't want language-specific parsing logic leaking into the investigation engine.

So I separated the language context layers.

The architecture became roughly:

                EdgeGuard
                    │
          Investigation Engine
                    │
        ┌───────────┼───────────┐
        │           │           │
      Java         C#       TypeScript
     Context      Context      Context
        │           │           │
       AST         AST         AST

The investigation logic can therefore remain shared while the language-specific context stays isolated.

This became an important design principle:

Share the investigation logic. Isolate the language-specific context.

Evidence Over Assumptions

At this point, I realized that simply generating a vulnerability report wasn't enough.

An AI can say:

"This might be vulnerable."

But what I really want is:

"Here is how you can reproduce it."

So EdgeGuard is designed to go beyond detection.

For potential vulnerabilities, the agent can attempt to construct counterexample inputs and generate runnable verification tests.

Depending on the language, that can mean:

• xUnit for C#
• JUnit for Java
• Mocha for TypeScript

The goal is to move from:

AI says:
"This might be vulnerable."

to:

AI hypothesis
      ↓
Counterexample
      ↓
Generated test
      ↓
Execution
      ↓
Evidence

The principle is:

Evidence over assumptions.

What I Learned

Building EdgeGuard changed how I think about AI-assisted code analysis.

The difficult part isn't simply getting an LLM to understand code.

The difficult part is controlling what the model is allowed to assume.

If an unknown helper is assumed safe, a vulnerability can disappear.

If every function is sent to an LLM, the system becomes expensive and difficult to scale.

If language-specific context is mixed together, supporting multiple languages becomes increasingly fragile.

So the architecture ended up combining three different approaches:

Static analysis for fast deterministic screening.

LLM investigation for reasoning about complex code paths.

Verification for turning hypotheses into evidence.

None of these approaches is perfect on its own.

Together, they are much more interesting.

Try to Break Your Own Code

EdgeGuard is an open-source VS Code extension, and it is still evolving.

The project is available here:

GitHub: https://github.com/phucphungbk/edgeguard

The idea behind EdgeGuard is deliberately simple:

Don't just ask AI to write your code. Ask it how your code could fail.

If you work with security-sensitive applications, large codebases, or legacy systems, I'd love to hear how you approach this problem.

How do you find edge cases?

How do you deal with false negatives in automated code analysis?

And most importantly:

How do you prove that an AI-generated security finding is actually real?

I'm building EdgeGuard to explore those questions.

Feedback, bug reports, benchmark results, and architectural criticism are all welcome.