This report explains how to design acceptance gates around approved plans, commit-specific evidence and independent review. It also identifies which controls are intended, partial or reported as operating, and why the available records do not support an end-to-end audit.
Generating changes is not the difficult part of building an internal software factory. Establishing a trustworthy basis for accepting them is.
An agent’s account of its own work is not evidence that the work is correct.
Why self-reporting is insufficient
During the early build phase, coding agents received work without a plan contract, acceptance gate or independent review. The implementing agent also reported what it had done.
Acceptance rested on that report rather than on evidence tied to the resulting software. This supported exploration of agent-led delivery. It was not sufficient for dependable engineering.
The acceptance model now separates:
- Approved plans and specifications
- Product clarification
- Implementation
- Commit-specific CI evidence
- Independent review
- Repair, rejection or acceptance
These controls are at different implementation stages. The records do not show them operating together on a single candidate change.
Planning is the contract
An agreed plan and specification provide the reference point for implementation and review. The plan defines:
- Intended behaviour
- Affected boundaries
- Validation approach
- Acceptance conditions
A technically precise plan can still contain an unresolved product decision. If ambiguity reaches implementation, an agent must stop or infer intent. That inference may produce coherent code that solves the wrong problem.
Product clarification therefore belongs before implementation. The specification records business rules, exceptional cases and acceptable trade-offs. A person decides the intended behaviour before code is written.
The target is not merely a plausible change. It is an approved interpretation of the requirement, implemented against conditions defined in advance.
Approved plans are not yet universal. Product decision records are also too limited to measure coverage.
Evidence must be independent and commit-specific
A plan defines the target. It does not prove that a candidate change reached it.
Continuous integration is intended to provide evidence from the required checks. That evidence counts only when independently verified and attached to the exact commit under consideration.
A test summary copied into an agent response is insufficient. It may:
- Refer to another revision
- Omit a failed check
- Describe an environment that cannot be reproduced
- Lack evidence that the required checks ran
A commit-bound evidence interface needs four defined elements:
| Element | Required definition |
|---|---|
| Identity | The immutable commit identity under consideration |
| Inputs | CI results, required attestations and provenance records |
| Validation | Checks that the evidence belongs to the commit and meets the required conditions |
| Failure states | Missing, malformed, incomplete, mismatched or non-compliant evidence |
The intended gate fails closed for each listed failure state. However, the records do not describe how commit identity is verified, how CI results are bound to that identity or how attestations are validated. They also provide no workflow evidence that acceptance is blocked.
Commit-bound evidence and fail-closed enforcement therefore remain design requirements rather than auditable controls.
Review is separate, but rejection policy is incomplete
The records state that the reviewing agent comes from a different model family from the implementing agent. They also state that findings are classified by severity and category.
Classification and rejection are separate controls. A workflow can classify findings without a complete policy for deciding which classifications block acceptance. The current records do not define:
- The severity taxonomy
- Rejection thresholds
- Findings that require escalation
- Findings that permit acceptance
- How multiple findings are combined
Severity classification is reported as operating, while rejection policy remains unspecified. Without workflow records or review artefacts, neither claim can be independently audited.
Cross-family review may reduce repeated assumptions and blind spots. This remains a hypothesis. No comparative evidence shows whether it finds defects that same-family review would miss.
Rejected work needs a defined exit
A capped repair budget is intended to prevent a candidate circulating indefinitely between implementation and review. When the budget is exhausted, the work should return for:
- Human judgement
- Replanning
- A change of scope
The records do not define how the cap is selected, configured or enforced. They also do not define the workflow state entered when the budget is exhausted.
The repair limit is therefore a design principle. It becomes an auditable control only when workflow definitions record the budget, transitions and terminal states.
The codebase is part of the control system
A strong workflow cannot compensate for an environment that agents cannot reproduce. Readiness requirements cover:
- Written engineering conventions
- A pinned runtime
- Dependency locking
- An architecture map
- Testing guidance
- Common failure modes
- Consistently invocable checks
The records identify weaknesses in agent tooling and default-branch protection. Repository policies or configurations supporting those findings are not available here.
The requirements also lack recorded enforcement methods. The available material does not show which checks are automated, which are reviewed manually or which block acceptance. Codebase readiness must therefore be treated as partial and not yet reproducibly assessed.
Every undocumented convention or environment-specific step pushes a decision back into inference. It also makes acceptance harder to audit.
How control status is assessed
This report uses four status terms:
| Status | Meaning |
|---|---|
| Intended | Required behaviour is described, but implementation evidence is unavailable |
| Partial | Some constituent behaviour is reported or required, but coverage, enforcement or supporting records are incomplete |
| Reported operational | The records state that the control operates, but linked configurations or artefacts are unavailable for independent verification |
| Auditable | Implementation records show the control acting on a specific candidate and commit |
No control can be marked auditable from the available material. The source records and repository policies needed to verify operational claims have not been supplied.
Current control status
| Control | Status | Basis |
|---|---|---|
| Agreed plan and specification | Partial | Plans provide a shared reference but are not universal |
| Product clarification before implementation | Partial | Human clarification is required, but durable decision records remain limited |
| Separate automated review | Reported operational | The records state that implementing and reviewing agents are separate; supporting workflow records are unavailable |
| Severity and category classification | Reported operational | Classification is reported, but the taxonomy and review artefacts are unavailable |
| Review rejection policy | Intended | Thresholds, escalation rules and rejection criteria are unspecified |
| Cross-family review | Reported operational | Different model families are reported as in use; effectiveness is unproven |
| Commit-bound CI evidence | Intended | Binding and verification mechanisms are not recorded |
| Attestation and provenance validation | Intended | Required attestations and validation checks are unspecified |
| Fail-closed acceptance gate | Intended | Failure conditions are stated, but enforcement evidence is unavailable |
| Capped repair budget | Intended | Configuration, enforcement and exhaustion transitions are unspecified |
| Codebase readiness controls | Partial | Requirements exist, but testing and enforcement methods are not recorded |
The missing delivery trace
A source-backed trace is needed to show one candidate passing through:
- Specification and plan approval
- Product clarification
- Implementation
- Identification of the candidate commit
- Commit-bound CI checks and attestations
- Independent review and classified findings
- Repair, rejection or escalation
- Final acceptance
Each stage must link the control to its corresponding artefact. Required artefacts include:
- The approved plan
- The product decision record
- The commit identifier
- CI evidence
- Attestations
- Review findings
- Repair history
- The acceptance decision
No such trace is available. Adding an example without those records would repeat the self-reporting problem that the acceptance model is intended to remove.
Measurement limits
The records do not support publication of figures for:
- Readiness
- Approved-plan coverage
- First-pass acceptance
- Verified-evidence coverage
- Delivery activity
Denominators, dates, reporting windows, work-item counts and eligibility rules are missing.
The records also do not establish a time saving, productivity multiplier or increase in individual working speed. Comparative data for cross-family and same-family review is unavailable.
What changes next
The remaining work is to turn stated requirements into recorded controls:
- Make approved plans universal
- Record product clarifications as durable decisions
- Define metric denominators, reporting windows and eligibility rules
- Publish the readiness checklist, enforcement method and scoring method
- Specify how CI evidence is bound to an immutable commit identity
- Define required attestations, provenance checks and validation failures
- Record an anonymised end-to-end delivery trace
- Define the severity taxonomy, rejection thresholds and escalation rules
- Establish a comparison method for cross-family review
- Define repair-budget configuration, enforcement and exhaustion states
- Provide workflow records and repository policies for reported controls
- Strengthen agent tooling
- Strengthen default-branch protection
What this means
The acceptance model separates implementation from planning, evidence and judgement. The available records establish design intent more clearly than implementation. Dependable acceptance requires reproducible evidence that every required control operated on the exact commit accepted.
Originally published at AFT Group Engineering Insights.
