Detect Breaking OpenAPI Changes in CI with a Zero-Dependency Python Diff

Breaking an API usually doesn't happen on purpose. Someone renames a field in a shared User schema, makes an optional request field required, or drops an old endpoint. The OpenAPI file changes by a few lines, the PR looks harmless, and a mobile client that's three versions behind starts failing. A plain text diff of openapi.json can't tell you which changes break clients. A semantic diff can.

SpecDiff is a single-file Python tool (standard library only) that compares two OpenAPI JSON documents and sorts every difference into BREAKING, WARNING or INFO. It knows direction: removing a field from a response breaks clients, while making a request field required breaks callers. With --fail-on-breaking, it exits with code 2 so CI can block the merge. Everything below was run locally.

Two versions of a spec

openapi.main.json is a small Users API with GET /users/{id} (returns #/components/schemas/User), POST /users (accepts NewUser) and DELETE /users/{id}/avatar.

The PR version, openapi.pr.json, does four things:

1. removes email from the User component schema;
2. makes name required in NewUser;
3. marks DELETE /users/{id}/avatar as deprecated;
4. adds GET /users.

Running it

python3 specdiff.py -b openapi.main.json -t openapi.pr.json --no-color --fail-on-breaking
echo "exit $?"

[SpecDiff Contract Verification Report]

Summary: 2 BREAKING | 1 WARNINGS | 1 INFO

[BREAKING] Property -> paths./users/{id}.GET.responses.200.application/json.schema.properties.email
         Property 'email' removed from response payload
[WARNING]  Deprecation -> paths./users/{id}/avatar.DELETE
         Operation DELETE /users/{id}/avatar has been marked as deprecated
[INFO]     Operation -> paths./users.GET
         HTTP Method GET added to /users
[BREAKING] Constraint -> paths./users.POST.requestBody.content.application/json.schema.required[name]
         Existing property 'name' marked as REQUIRED
exit 2

Both breaking changes were made inside components/schemas, not in the path definitions. SpecDiff follows the local $ref pointers (#/components/...) and reports each finding at the endpoint where clients will notice it, which is the useful place for a reviewer.

A purely additive change (a new optional avatar_url field and a new GET /users) passes:

Summary: 0 BREAKING | 0 WARNINGS | 2 INFO

[INFO]     Operation -> paths./users.GET
         HTTP Method GET added to /users
[INFO]     Property -> paths./users/{id}.GET.responses.200.application/json.schema.properties.avatar_url
         Property 'avatar_url' added (optional)
exit 0

What it checks

From the rules in the source:

Reports for PRs and tooling

-f markdown writes a report you can post as a PR comment:

**Status**: ❌ CONTRACT BREACH DETECTED

| Metric | Count |
| :--- | :--- |
| **Breaking Changes** | 2 |
| **Warnings / Deprecations** | 1 |
| **Informational** | 1 |

### 🚨 Breaking Changes
| Category | Target Path | Issue Description |
| :--- | :--- | :--- |
| `Constraint` | `paths./users.POST.requestBody...required[name]` | Existing property 'name' marked as REQUIRED |
| `Property` | `paths./users/{id}.GET.responses.200...properties.email` | Property 'email' removed from response payload |

-f json gives a summary plus one object per finding, with the old and new value:

{
  "summary": {"total_diffs": 4, "breaking": 2, "warning": 1, "info": 1},
  "divergences": [
    {"severity": "BREAKING", "category": "Property",
     "path": "paths./users/{id}.GET.responses.200.application/json.schema.properties.email",
     "message": "Property 'email' removed from response payload",
     "old_value": "email", "new_value": null}
  ]
}

In CI

Compare the spec on the PR branch with the one on main:

- name: API contract check
  run: |
    git fetch origin main --depth=1
    git show origin/main:openapi.json > /tmp/openapi.main.json
    python3 tools/specdiff.py -b /tmp/openapi.main.json -t openapi.json \
      --fail-on-breaking --no-color

If there's a breaking change the step fails with exit 2. Without --fail-on-breaking the report is informational and the exit code is 0. When a breaking change is intentional (a new major version), skip the check for that PR or compare against the new version's baseline.

Try it without your own spec

python3 specdiff.py --demo

The demo runs the engine against two built-in sample specs, so you can see the output format first.

Limitations

• JSON only. For YAML specs, convert first, e.g. yq -o=json openapi.yaml > openapi.json.
• Only local references (#/components/...) are resolved. Specs split over several files need bundling into one first (most OpenAPI tooling has a "bundle" command).
• oneOf / anyOf / allOf compositions and enum value changes aren't analysed.
• Constraint changes like maxLength, pattern or minimum aren't classified.
• It reports changes. It doesn't check that your implementation matches the spec.

FAQ

OpenAPI 3.0 or 3.1? It reads the standard paths / components structure that both share. Swagger 2.0 (definitions) isn't the target.

Dependencies? None. Python 3 and the standard library.

Can I hide colours in logs? Yes, --no-color.

SpecDiff is a one-time purchase that includes the source and README: get SpecDiff ($19). More details are on the SpecDiff product page.