TL;DR
We encrypted every competition submission and required a valid team signature. Yet another GitHub user could copy an unchanged public bundle and spend the original team’s daily quota.
This postmortem explains why the cryptography worked, why the system still failed, and why secure public workflows must authenticate state transitions rather than artifacts alone.
Postmortem status: The workshop has ended, and the affected scoring workflows have been manually disabled. There is no active scoring endpoint associated with this design.
How a valid digital signature became a transferable authorization token in a public GitHub competition
We encrypted the submission.
We signed its manifest.
We verified the signature against a team certificate.
The signature was valid.
And yet, another GitHub user could copy the complete submission into a new pull request and make the original team spend another scoring attempt.
No private key had to be stolen.
No signature had to be forged.
No ciphertext had to be decrypted.
The cryptography did exactly what we asked it to do.
The problem was that we had asked the wrong security question.
A GitHub-native data-cleaning tournament
In May 2026, we ran a data-cleaning tournament as part of a Python workshop.
Each team wrote a Python cleaner that transformed public training and test feature files. The official scorer then trained a fixed model, evaluated the cleaned test data against hidden labels, and recorded the result on a shared leaderboard.
We wanted several properties:
- teams should not be able to inspect the hidden test labels;
- organizers should be able to execute each cleaner consistently;
- teams should not need to reveal their cleaner source code publicly;
- only an authorized team should be able to submit under its assigned team identity;
- each team should receive only a limited number of scored attempts per day.
GitHub already provided most of the surrounding infrastructure:
- participant identity;
- forks and pull requests;
- workflow events;
- hosted runners;
- workflow artifacts;
- an auditable repository history;
- GitHub Pages for the leaderboard.
So we used pull requests as the submission transport.
The system looked approximately like this:
Team writes clean.py
│
│ encrypt to organizer certificate
▼
submission/clean.py.cms
│
│ SHA-256
▼
submission/manifest.json
│
│ sign with team private key
▼
submission/manifest.sig
│
│ public pull request
▼
GitHub Actions
│
├── verify manifest signature
├── decrypt cleaner
├── execute cleaner
├── encrypt cleaned outputs
└── pass outputs to trusted scorer
│
├── decrypt hidden labels
├── train fixed model
├── calculate score
└── update leaderboard
An official submission contained the encrypted cleaner, the manifest, its detached signature, and the dependency files required to execute the cleaner.
The repository’s helper script encrypted clean.py, calculated the ciphertext’s SHA-256 digest, placed that digest in the manifest, and signed the manifest using the team’s private key.
At first glance, this appeared to provide a clean chain of trust.
What exactly did the signature authenticate?
A simplified manifest looked like this:
{
"team_id": "team-06",
"code_path": "submission/clean.py.cms",
"code_sha256": "b7a4...64 hexadecimal characters..."
}
The verification script performed several sensible checks:
- validate the format of
team_id; - require the code path to be
submission/clean.py.cms; - calculate the actual SHA-256 digest of the encrypted cleaner;
- require that digest to match
code_sha256; - look up the team’s certificate in an allowlist;
- validate that certificate against the tournament CA;
- verify the detached signature over the manifest.
The workflow then installed the submitted dependencies, decrypted the cleaner, ran it against the tournament data, validated its outputs, and uploaded encrypted scoring artifacts.
The verifier was therefore able to establish:
The holder of Team 06’s private key signed this exact manifest, and the encrypted cleaner currently present has the digest recorded in that manifest.
That statement was true.
But it was narrower than the statement we implicitly relied upon:
Team 06 is authorizing this particular GitHub pull request as a new scoring attempt.
Those are not the same statement.
The missing subject: who is presenting the signed artifact?
A digital signature binds a key to some bytes.
It does not automatically bind those bytes to every context in which they might later appear.
Our signed manifest included:
- team identity
- encrypted-code path
- encrypted-code digest
It did not include, or otherwise bind itself to:
- the current GitHub PR author
- the author’s immutable GitHub user ID
- the source repository
- the pull request number
- the current head commit
- a unique logical attempt ID
- an expiry time
- a one-time challenge
More importantly, the workflow did not independently compare the current PR actor or source repository with an identity authorized to submit for the signed team_id.
The workflow took the submitted files from the pull request’s head repository and verified the team manifest, but the verification decision remained entirely about the artifact. It did not ask whether the GitHub user presenting that artifact was entitled to act for the team named inside it.
In effect, our valid signed bundle had become a bearer artifact:
Anyone possessing an unchanged copy could present it, while the system continued attributing the action to the original signing team.
The signature was public evidence of past authorization.
We had accidentally treated it as reusable authority for a new action.
The Threat Model (Replay Attack)
Consider two participants:
Alice — a legitimate member of Team A
Charlie — a different GitHub user
Alice prepares a valid submission:
submission/clean.py.cms
submission/manifest.json
submission/manifest.sig
Her manifest says:
{
"team_id": "team-a",
"code_path": "submission/clean.py.cms",
"code_sha256": "H"
}
Team A’s private key signs that manifest.
Alice opens a pull request.
The submission is validly scored.
Because the pull request belongs to a public repository, Charlie can obtain the submitted encrypted cleaner, manifest, signature, and associated public dependency files.
Charlie does not edit them.
He does not need to understand the cleaner.
He simply places the same files in another branch or fork and opens another pull request.
The verifier now observes:
Manifest signature: valid
Team certificate: valid
Ciphertext SHA-256: matches
Team ID: team-a
Every cryptographic check succeeds, because the signed artifact has not changed.
The trusted scorer subsequently obtains team_id from that manifest. It counts the day’s previous leaderboard submissions carrying the same team ID, checks the configured daily limit, and appends each accepted result as another submission for that team.
Therefore Charlie’s replay is charged to Team A.
Repeated replays can consume Team A’s available daily attempts.
After that, Alice may be unable to submit another scored attempt that day.
The problem was not duplicate content
This distinction matters.
It may be perfectly legitimate for Alice to submit exactly the same cleaner twice.
Perhaps she wants to confirm reproducibility. Perhaps she accidentally changed something elsewhere. Perhaps competition policy simply says that every intentional submission consumes one attempt, even when the code is identical.
The desired policy may be:
Alice intentionally submits content H as attempt A-001
→ accepted and charged to Alice’s team
Alice intentionally submits content H again as attempt A-002
→ accepted and charged again to Alice’s team
The vulnerability was not that identical content could ever be processed more than once.
The vulnerability was:
Charlie could make that decision on Alice’s behalf.
A secure redesign therefore should not necessarily deduplicate by file hash.
It should distinguish between:
- content identity — whether two submissions contain the same files;
- attempt identity — whether they represent the same logical submission attempt;
- principal identity — who is requesting the attempt;
- team attribution — which team the authenticated principal belongs to. An explicit, signed attempt identifier can allow Alice to resubmit the same content intentionally, while still making workflow retries idempotent.
Binding the request to Alice’s authenticated GitHub identity prevents Charlie from replaying Alice’s signed attempt.
Why stronger encryption would not have fixed it
It is tempting to look at a replay involving an encrypted file and conclude that the encryption mechanism needs to be strengthened.
But confidentiality was not the failed property.
Charlie never learned the contents of the encrypted cleaner.
Changing the encryption algorithm would not stop him from copying the exact ciphertext.
Authenticated encryption can detect modification. It does not, by itself, prevent an unchanged valid ciphertext from being presented again.
Likewise, the digital signature correctly prevented Charlie from changing:
team-a
into:
team-charlie
Any such modification would invalidate Team A’s signature.
But Charlie did not need to modify anything.
Replay attacks live in the gap between:
This message is authentic.
and:
This message is authorized here, now, by the actor currently presenting it.
Cryptographic authenticity is one input to authorization. It is not the complete authorization decision.
Public multi-tenancy changes the threat model
In many multi-tenant systems, tenants are separated by confidentiality boundaries.
Tenant A cannot inspect Tenant B’s records, credentials, or requests.
Our environment was different.
Every team interacted through the same public repository. Pull requests were visible. Signed artifacts were visible. Encrypted submissions were visible. Tenants could inspect and copy one another’s transport-level records.
This was not an accidental data leak. Public visibility was part of the platform model.
That leads to a useful design rule:
In a publicly observable multi-tenant system, every public artifact must be assumed copyable and replayable.
Security cannot depend on another tenant being unable to obtain:
a manifest
a public key
a signature
an encrypted bundle
a previous request
Instead, each proposed action must be evaluated using both:
the submitted artifact
+
the authenticated context in which it is being presented
For GitHub, that context may include the event’s immutable actor ID, repository ID, pull request metadata, a protected identity registry, and the current authoritative system state.
The root cause in one sentence
The root cause was:
We treated possession of a valid signed artifact as authorization to initiate a new state transition.
The signature answered:
Who signed these bytes?
The scorer needed answers to several additional questions:
Who is presenting them now?
Is that GitHub identity registered to the signing key?
Does that identity belong to the claimed team?
Is this a new intentional attempt or a retry of an existing attempt?
Is the requested transition allowed under the team’s current quota?
Which authoritative state should be updated?
Those questions belong to an authorization and state-management layer, not to the signature algorithm alone.
Operational response
By the time of this publication, the workshop had ended and the relevant cleaner and scoring workflows had been manually disabled. GitHub’s workflow pages show the affected workflows as disabled, so the historical design is no longer an active scoring surface.
We are publishing this as a postmortem rather than as a vulnerability in GitHub.
GitHub Actions behaved according to its configuration.
The design error existed in our application-level trust model:
public pull requests
+
reusable signed artifacts
+
team-based quotas
+
no binding between the current actor and the signed team identity
From signed artifacts to authenticated state transitions
The redesigned system begins with a different abstraction.
A pull request does not directly become a registration or submission.
Instead, it is an untrusted proposal for a state transition.
Public PR artifact
+
Authenticated GitHub actor
+
Registered participant key
+
Immutable team membership
+
Current quota and attempt state
↓
Policy verification
↓
Authorized state transition
The important question is no longer only:
Is this signature valid?
It becomes:
Is this authenticated actor authorized to request this exact transition in the system’s current state?
The next part of this series will examine the first half of that redesign:
- self-service participant onboarding;
- binding a GitHub identity to a participant public key;
- allowing organizers to accept previously unknown participants;
- requiring every member to consent to the same complete team manifest;
- making team membership immutable for the duration of the competition;
- storing the resulting identity and team records as auditable authoritative state.
The final part will return to submissions:
- actor-bound submission manifests;
- explicit attempt identifiers;
- intentional resubmission of identical content;
- cross-principal replay prevention;
- quota reservation;
- concurrency;
- and scoring attribution.
Closing thought
The most important lesson was not about a particular encryption mode or signature algorithm.
It was this:
A valid signature proves that a key endorsed some bytes. It does not prove that the person presenting those bytes now is entitled to trigger a new action.
In a system where every tenant can see every other tenant’s artifacts, signed objects should be treated as public claims—not as bearer authority.
Authentication, authorization, freshness, identity binding, and state transition policy must still happen at the moment the artifact is used.
