This article was originally published on BuildZn.

Everyone talks about static analysis for security, but honestly, most tools choke on subtle cryptographic flaws, especially side-channels. Spent a week trying to prove this point, building intentionally broken AES, then throwing it at both commercial tools and Claude 3.5 Sonnet. The results from the Claude AI crypto vulnerability scan were wild; it caught stuff the others just glossed over.

Why Traditional Scanners Miss Subtle Crypto Flaws & The Claude AI Crypto Vulnerability Scan

Look, static analysis tools are great for boilerplate, SQLi, XSS, you name it. But when it comes to crypto, they often fall flat. They see patterns, sure, but they struggle with contextual reasoning—like how different error messages or slight timing variations can leak critical info. That's where LLMs like Claude come in, offering a new angle for LLM cryptographic analysis.

I've shipped 20+ apps, built AI systems like FarahGPT and NexusOS. Trust me, I've seen enough crypto implementations to know even experienced devs make subtle mistakes. Traditional tools might flag a missing IV or hardcoded key (obvious stuff), but they rarely connect the dots between an error handler's output and a potential padding oracle attack. This gap is exactly what I wanted to test with a Claude AI crypto vulnerability scan.

Here's the thing — LLMs, especially advanced ones like Claude 3.5 Sonnet, can reason about code. They can infer attacker intent, analyze error paths, and even spot potential timing differences based on execution flow. This is a game-changer for AI security testing tools, moving beyond pattern matching to actual vulnerability detection based on logical inference.

The Setup: My Intentionally Flawed AES Implementation

To put Claude to the test, I cooked up a Python class, FlawedAES, using cryptography library version 42.0.7. This isn't some toy example; it's designed to mimic common, subtle mistakes I've seen in real-world code where developers might differentiate error handling.

The goal: Create a padding oracle vulnerability and a related side-channel leakage through error messages.

import os
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import padding
from cryptography.exceptions import InvalidTag # For GCM, not used here but good to know

class CryptoError(Exception):
    """Custom exception for cryptographic errors."""
    pass

class FlawedAES:
    def __init__(self, key: bytes, iv: bytes):
        if len(key) != 32:
            raise ValueError("AES-256 key must be 32 bytes.")
        if len(iv) != 16:
            raise ValueError("AES-CBC IV must be 16 bytes.")
        self.key = key
        self.iv = iv

    def encrypt(self, plaintext: bytes) -> bytes:
        padder = padding.PKCS7(algorithms.AES.block_size).padder()
        padded_data = padder.update(plaintext) + padder.finalize()
        cipher = Cipher(algorithms.AES(self.key), modes.CBC(self.iv), backend=default_backend())
        encryptor = cipher.encryptor()
        ciphertext = encryptor.update(padded_data) + encryptor.finalize()
        return ciphertext

    def decrypt_data(self, ciphertext: bytes) -> bytes:
        """
        Decrypts ciphertext. Intentionally flawed to demonstrate padding oracle and side-channel.
        """
        try:
            cipher = Cipher(algorithms.AES(self.key), modes.CBC(self.iv), backend=default_backend())
            decryptor = cipher.decryptor()
            # Step 1: Decrypt raw ciphertext
            decrypted_padded_data = decryptor.update(ciphertext) + decryptor.finalize()

            # Step 2: Unpad the data
            # --- INTENTIONAL FLAW START ---
            # This try-except block specifically catches padding errors,
            # providing a distinct error path and message.
            try:
                unpadder = padding.PKCS7(algorithms.AES.block_size).unpadder()
                unpadded_data = unpadder.update(decrypted_padded_data) + unpadder.finalize()
                return unpadded_data
            except ValueError as e:
                # This specific ValueError indicates a padding issue.
                # An attacker could distinguish this error from other decryption failures.
                # In a real system, the *timing* of this exception vs. a broader one
                # could also be a side channel.
                print("DEBUG: Padding error detected specifically.") # For debugging analysis
                raise CryptoError("PaddingError: Invalid PKCS7 padding found.") from e
            # --- INTENTIONAL FLAW END ---

        except Exception as e:
            # This catches all other decryption-related errors (e.g., malformed ciphertext not related to padding structure).
            # This path is intentionally generic and distinct from the padding error path.
            print("DEBUG: Generic decryption error detected.") # For debugging analysis
            raise CryptoError("DecryptionFailed: Generic decryption error occurred.") from e

# Example usage (not part of the audit target)
# key = os.urandom(32)
# iv = os.urandom(16)
# aes_flawed = FlawedAES(key, iv)
# encrypted = aes_flawed.encrypt(b"my secret message")
# decrypted = aes_flawed.decrypt_data(encrypted)

The core issue lies in decrypt_data. Notice the nested try...except blocks. The inner one specifically catches ValueError from the unpadder, signalling an invalid padding. The outer except Exception catches everything else, leading to a generic DecryptionFailed error. This distinction is subtle but deadly.

Claude's First Pass: Identifying the Padding Oracle with Specific Prompts

My first prompt was direct, no fluff. I gave it the FlawedAES class and asked for crypto vulnerabilities. I also mentioned the cryptography library version 42.0.7 because library-specific behaviors can sometimes be relevant to vulnerabilities.

Prompt 1:

Here's a Python class `FlawedAES` using the `cryptography` library version `42.0.7`. Review the `decrypt_data` method for cryptographic vulnerabilities, especially padding oracles, timing attacks, or any other weaknesses. Assume `key` and `iv` are properly generated and kept secret. Provide specific line numbers for identified issues and explain the exploit.

python

(Paste the full FlawedAES class code here)

plaintext

Claude 3.5 Sonnet's response was sharp. It immediately honed in on the padding oracle:

Claude 3.5 Sonnet Analysis (Excerpt):

Vulnerability: Padding Oracle Attack

Affected Method: decrypt_data
Relevant Lines:

• L38: try: (inner try block around unpadder)
• L41: except ValueError as e:
• L46: except Exception as e:

Explanation:
The decrypt_data method is vulnerable to a padding oracle attack. The critical flaw is the differentiation in error responses based on whether the padding is valid or not.

When unpadder.finalize() encounters invalid PKCS7 padding, it raises a ValueError. This specific ValueError is caught by the inner except ValueError block (L41), which then raises a CryptoError("PaddingError: Invalid PKCS7 padding found.").

However, if the ciphertext is corrupted in a way that is not a padding issue (e.g., a bit flip that results in a syntactically valid but incorrect padding, or an error prior to the unpadding step), it might raise a different exception or none at all, eventually falling to the outer except Exception block (L46), which raises CryptoError("DecryptionFailed: Generic decryption error occurred.").

An attacker can leverage this distinction. By sending slightly modified ciphertexts and observing whether they receive a "PaddingError" versus a "DecryptionFailed" or another generic error, they can iteratively determine if the last block's padding bytes are valid. This allows them to decrypt the ciphertext byte by byte without knowing the key.

This is a classic padding oracle. The print statements (L43, L48) themselves aren't the vulnerability, but they highlight the distinct execution paths that lead to different error messages.

That's exactly it. Claude didn't just say "padding oracle." It pinpointed the mechanism—the distinct error messages resulting from different exception handling paths. Most static analyzers would just flag the broad except Exception or maybe a generic "potential padding issue" without explaining how the error differentiation specifically enables the oracle.

Uncovering Side-Channel Leakage: Claude's Deeper Dive into Error Handling

Identifying the padding oracle was good, but I pushed Claude further on the AI security testing tools angle. I wanted to see if it could reason about side-channel leakage beyond just the explicit error message. This is often missed even by experienced human auditors because it requires deep understanding of execution paths and subtle timing variations.

Prompt 2:

You correctly identified a padding oracle in `decrypt_data` in `FlawedAES` using `cryptography` library version `42.0.7`. Now, can you elaborate on potential *side-channel leakages* related to the error handling within that function? Specifically, analyze the execution flow and potential timing differences between the `try...except ValueError` block (L38-L43) and the outer `except Exception` block (L46-L48). Would a difference in *response timing* between `PaddingError` and `DecryptionFailed` pose a risk, in addition to the distinct error messages? Consider a scenario where an attacker can supply arbitrary ciphertexts and measure the exact response time of the API endpoint calling this function.

Claude's response here delivered on the unique claim:

Claude 3.5 Sonnet Analysis (Excerpt):

Vulnerability: Side-Channel Leakage (Timing Attack based on Error Paths)

Affected Method: decrypt_data
Relevant Lines:

• L38-L43 (Padding error path)
• L46-L48 (Generic decryption error path)

Explanation:
Yes, absolutely. Beyond the distinct error messages, a difference in response timing between the PaddingError path and the DecryptionFailed path constitutes a critical side-channel leakage, exacerbating the padding oracle vulnerability.

Execution Flow and Timing Differences:

1. PaddingError Path (L38-L43):

   • The decryptor.update() + decryptor.finalize() (L34) must complete successfully.
   • Then, the unpadder.update() + unpadder.finalize() (L39) is attempted.
   • If padding is invalid, a ValueError is raised immediately by the unpadder.
   • This ValueError is caught by the inner except block (L41), leading to CryptoError("PaddingError...").

2. DecryptionFailed Path (L46-L48):

   • This path is typically hit if an error occurs before or during the initial decryption (L34), or if an unexpected exception occurs during unpadding that isn't a ValueError (though less likely with PKCS7).
   • Errors during initial decryption (e.g., malformed block size, internal cryptography library issues not related to padding structure) would likely lead to this path.
   • The critical observation is that the code executed within the try block leading to a ValueError for padding might be longer or shorter than the code path leading to a general Exception.
   • Specifically, if the decryption and an attempted unpadding (which then fails) takes slightly longer than a decryption that fails immediately due to a malformed block, an attacker can distinguish these cases.

Risk Assessment:
An attacker performing a Claude security auditing simulation could, with high precision, measure the response time of the API endpoint.

• If a request takes T1 milliseconds and returns "PaddingError", it signals that the decryption succeeded but padding failed.
• If a request takes T2 milliseconds and returns "DecryptionFailed", it signals a different type of error.

Even if T1 and T2 differ by only a few microseconds, in a high-throughput system or with sufficient samples, this timing difference can be statistically significant enough to distinguish valid padding attempts from other decryption failures. This enhances the padding oracle, allowing an attacker to determine padding validity not just by the error message, but by the subtle timing difference in the error response.

This particular flaw is often missed by static analysis tools because they don't execute the code or model CPU cycles. They see two distinct except blocks but don't infer the runtime implications of different execution depths and exception handling overheads.

This is the money shot. Claude didn't just state the timing attack; it broke down the specific execution flow differences that lead to it. It articulated how the unpadder doing its work before failing creates a different timing profile than a general decryption failure. A leading static analysis tool like CodeQL or SonarQube would flag the broad except Exception or maybe a "potential for information leakage through error messages" but would not provide this level of detail on the timing implications of the distinct error handling paths. This is a fundamental difference in reasoning.

What I Got Wrong First

Honestly, when I first started using LLMs for vulnerability detection LLM tasks, I was too generic with my prompts. I'd just dump a bunch of code and say, "Find security bugs." Claude would spit out a decent list, but it often felt like a glorified linter, hitting the obvious stuff.

My big mistake was not being specific enough about the types of attacks I was looking for, or the context I wanted it to analyze. For crypto, "find vulnerabilities" isn't enough. You need to guide it towards "padding oracle," "timing attacks," "side-channels," and even specify the attacker's capabilities (e.g., "attacker can control input and observe error messages/timing").

Turns out, prompt engineering for security is key. The more context you give, the deeper the analysis. It’s like asking a junior dev versus a seasoned security expert. If you just say "code review," you get basic stuff. If you say, "review this for padding oracles assuming an active network attacker can manipulate ciphertexts and observe response times," you get a whole different level of insight.

FAQs

Can LLMs like Claude replace traditional static analysis for crypto?

No, not directly. LLMs are powerful supplements. Static analysis excels at broad pattern matching and policy enforcement across large codebases. LLMs, with their reasoning capabilities, can identify subtle logical flaws, side-channels, and complex attack vectors that static tools often miss. Use them together, not in isolation.

How accurate is Claude for cryptographic analysis compared to human experts?

Claude 3.5 Sonnet demonstrates remarkable accuracy in identifying specific crypto flaws like padding oracles and reasoning about side-channels, often matching the detail a human expert would provide. However, human experts still have an edge in understanding the broader system context, threat models, and real-world exploitability nuances. LLMs are fantastic at augmenting, not replacing, human expertise.

What's the best way to prompt Claude for security audits?

Be specific. Don't just ask "find bugs." Specify the type of vulnerability (e.g., XSS, SQLi, padding oracle), the attacker's capabilities (e.g., "active network attacker," "local privilege escalation"), and any relevant environmental factors (e.g., "assuming this code runs on AWS Lambda"). Provide full, runnable code snippets. The more precise your prompt, the better and more targeted Claude's vulnerability detection LLM analysis will be.

Look, I'm not saying throw out your SAST tools. But relying solely on them for crypto security is a mistake. The fact that Claude 3.5 Sonnet, with a well-crafted prompt, could articulate a padding oracle and its accompanying timing side-channel with this level of detail—a flaw often missed by traditional static analysis—proves that LLMs are a serious tool in the security engineer's kit. They're not a magic bullet, but they're a damn good magnifying glass for the flaws we usually squint to find. Use them.