capsurface: reviewing capability changes in npm dependencies

A dependency bot opens a PR. One version changes, the tests pass, and the
changelog says “maintenance.”

You could read the package diff. You could also finish the thing you were doing before the bot opened twelve PRs.

I built capsurface to make that review easier. It scans package source for filesystem, network, process and credential access, then compares the results against a baseline committed to your repository. When an update adds something that needs review, the report includes the file, line and reason for blocking it.

Version 0.1.0 is available on npm. The scanner runs offline, has zero required
dependencies and is MIT licensed. It cannot supply the afternoon you would
need to audit every dependency, but it can give you somewhere to start.

A patch release with something extra

The repository includes two versions of a fictional package, handy-color-utils.
Version 2.3.0 converts hex colors to RGB. Version 2.3.1 still converts hex colors
to RGB, but has developed an interest in your npm credentials.

The public API is unchanged. The extra work happens in a new postinstall script
that reads files and uses network and process APIs. The package continuing to
do its original job is part of what makes a change like this easy to overlook.

Here is a selection of what the comparison reports:

You can run this example with Node.js and Bash:

git clone --branch v0.1.0 --depth 1 https://github.com/VictorMartins3/capsurface.git
cd capsurface
bash examples/run-demo.sh

The demo scans both versions and compares them. The check returns exit code 1;
the demo script treats that expected failure as success. It reads the fixture
files without executing their code.

This is a synthetic example. It gives you a way to inspect the output and
change the inputs yourself, rather than taking a screenshot on trust.

Why keep a baseline?

A package using https is usually unremarkable. A package gaining access to
https after an update gives you a specific change to examine. Perhaps it now
fetches a binary. Perhaps a previously local operation sends data to a server.
Either way, there is source to look at and a decision to make.

capsurface records the observed capabilities in capsurface.lock.json. The
baseline belongs in version control so a reviewer can see what was accepted.
Generating the file takes a command; reviewing what goes into it takes longer.
Committing the JSON does not retroactively perform that review.

The scanner looks at supported source files throughout a package, including
code under test/ and docs/, as well as installation scripts. Directory names
do not determine when code can run. The event-stream incident
is a useful example: malicious code loaded encrypted data disguised as a test
fixture during the targeted build. Blocking installation scripts alone does
not inspect code that an application later imports.

That is a reason to inspect dependency content. It is not evidence that
capsurface would have caught event-stream: I have not reproduced that attack
against its original artifacts, and encrypted payloads can hide the indicators
this scanner looks for.

Using it on a project

Install the CLI separately from the project you want to inspect:

npm install --global --ignore-scripts capsurface@0.1.0

In that project, install dependencies with scripts disabled and scan the tree:

npm ci --ignore-scripts
capsurface scan-tree node_modules --out .capsurface/manifests

Inspect the manifests before accepting them as your starting point:

capsurface baseline .capsurface/manifests --out capsurface.lock.json

Commit capsurface.lock.json. Keep the generated manifests and reports out of
Git. For a later dependency update, repeat the installation and scan, then run:

capsurface review .capsurface/manifests --baseline capsurface.lock.json --out review.md

The Markdown report includes blocking reasons and source locations. A blocked
review returns 1 while still writing the report. Add --lockfile package-lock.json
to include npm dependency origins, or --fail-on-new to require approval for
new packages too. JSON and SARIF output are also available.

There is a GitHub Action and example workflow
for running this on PRs, including Dependabot and Renovate updates. The report
appears in the workflow job summary. Start with report-only: 'true' to see how
much review your dependency updates need. Invalid or incomplete scans still
fail; report-only applies to the policy findings.

One detail matters here: a PR can change both a dependency and its baseline.
The Action retains the comparison against the target branch's baseline, then
checks the proposed baseline separately. Accepting a change should not remove
it from the reviewer's view.

Accepting one package's changes

If an update is expected, you can approve the installation identified by the
report instead of regenerating the whole baseline:

capsurface approve .capsurface/manifests --baseline capsurface.lock.json \
  --id <review-id> --reason "Reviewed the new HTTP client"

Replace <review-id> with the ID from the report. The approval covers that
installation's observed changes and binds them to its version and file content.
Other packages stay pending. Approvals can have an expiration, and a stale
review or incomplete scan cannot be approved.

Approving one installation leaves the remaining findings visible. It helps
avoid the familiar debugging technique of updating the expected result until
the test agrees with you. Someone still needs to read the change and explain
why it is acceptable.

Analysis and its limits

The default scanner uses source-text heuristics. Experimental --deep adds
optional Acorn and acorn-typescript parsers to resolve supported aliases,
loader forms and static expressions. It also adds detail about operations
such as process execution and environment enumeration. Neither mode executes
the package.

You can also scan before installation with scan-lock. It takes local tarballs
mapped to an npm v2/v3 lockfile, verifies their integrity and scans their
contents. Downloading them is a separate step. This first version rejects
workspace links, Git/local dependencies and bundled dependency trees; its
baselines are separate from installed-tree baselines.

There are straightforward ways for a capability diff to miss malicious
behavior. A package that already reads credentials and makes network requests
could misuse those capabilities without adding a new category. An encrypted
payload may conceal its behavior. Finding a credential read and a network call
in the same file does not prove that one sends data to the other.

Benign updates also add capabilities, so some findings will require review
without being security problems. capsurface does not enforce runtime
permissions or establish that a package is safe.

The verification notes
record the test inputs, scanner revisions and coverage limits. For example,
archive scanning was compared with independent system-tar extraction across
28 published tarballs containing 2,488 source files. The manifests and content
hashes matched on those inputs. That checks the archive path; it is not a
malware detection rate.

Trying 0.1.0

I'd like feedback on the review itself: can you understand why an update was
blocked, find the relevant source and decide what to approve? An unclear
report is a useful bug. So is a small fixture that demonstrates a missed or
incorrectly attributed capability.

The repository has the demo,
CLI documentation and issue tracker. Use the security policy
for security-sensitive reports.

Install from npm ·
0.1.0 release notes