Pressure-testing Ota on cloud.devenv.sh with Nix and devenv

Overview

cloud.devenv.sh is a useful Ota pressure case because its execution path crosses three distinct
ownership boundaries:

• the host provides nix
• Nix launches a pinned devenv revision
• devenv owns the repository's test and up subcommands

A weak contract would flatten that into an opaque shell command. The stronger model preserves who
owns each layer and lets Ota select the repository operation without pretending to own devenv's
internal process semantics.

The immutable hosted matrix at repository revision
29c44c235795d3af3b1b7eafdaacd63a416d1f06
completed successfully on Ubuntu and macOS with released Ota v1.6.26. The result proved the
modeled boundary, but it also exposed an important limit: a green orchestrator exit is not proof
that every nested process succeeded.

Evidence snapshot

The hosted workflow run
used:

• cloud.devenv.sh revision 29c44c235795d3af3b1b7eafdaacd63a416d1f06
• Ota v1.6.26 source revision 08855fd1b7269abe9342dc9dd45acf4ed1885063
• contract-pinned devenv launcher revision 65a7c40d185350f0e96783b3dd8d4afab3bb7034
• Ubuntu binary SHA-256 3d545be8eb9701542284d28bb89fba83c77ae30c1d1157d9dd87fb83d961e05e
• macOS binary SHA-256 a2448590b0fedfeaffd29535103d5209856e3b0fcb42a93d0f47256920b58a79

Both jobs validated the contract with zero errors and warnings, inspected Doctor's deterministic
fix preview, listed the public task surface, exported execution topology, and dry-ran test,
verify, dev, and the verify workflow. The contract intentionally declares no agent-safe task,
and the retained output confirmed that ota tasks --safe --json returned an empty list.

Doctor did not call the repository ready. Its retained fix preview reported verdict: risky and
agent_verdict: not_ready because .ota/state/, .ota/contracts/, .ota/receipts/, and
.ota/proof/ were not ignored by Git. The workflow intentionally accepted that advisory posture;
its green status must not be read as an agent-readiness verdict.

The Ubuntu job additionally executed the finite test task through Ota. The macOS job did not run
that real task; it proved only validation, discovery, topology, Doctor, and dry-run admission.

The contract boundary

The contract keeps launcher ownership explicit:

orchestrators:
  devenv:
    kind: devenv
    required: true
    config_files:
      - devenv.nix
    launcher:
      exe: nix
      args:
        - run
        - github:cachix/devenv/65a7c40d185350f0e96783b3dd8d4afab3bb7034#devenv
        - --

The repository's verification command remains a direct devenv subcommand:

tasks:
  test:
    command:
      exe: test
    execution:
      orchestrator:
        ref: devenv
        mode: subcommand

The long-running development path uses the same owner without being mislabeled as finite work:

tasks:
  dev:
    launch:
      kind: command
      exe: up
    execution:
      orchestrator:
        ref: devenv
        mode: subcommand

This separation gives Ota enough structure to explain and select the path while leaving devenv
responsible for the process graph it launches.

What the hosted matrix proved

The retained artifacts and logs prove that:

• released Ota v1.6.26 loaded and validated the immutable contract on Ubuntu and macOS
• Ota resolved nix as the host requirement and the pinned devenv invocation as orchestrator truth
• task discovery preserved finite test, aggregate verify, and long-running dev as distinct lanes
• Doctor preserved the repository-hygiene warning and not_ready agent verdict
• all advertised native lanes admitted dry-run on both operating systems
• the Ubuntu job executed ota run test --native --stream and Ota reported success after devenv returned exit code zero
• the Ubuntu run stopped the devenv-managed processes before Ota emitted its successful run summary

This is execution evidence for one selected Ubuntu task. It is not runtime proof, lifecycle proof,
or evidence that every repository behavior was governed.

The green run exposed a real boundary

During the Ubuntu task, devenv logged that backend-migrate failed because
cloud.devenv.toml was missing. Other work continued, the frontend build completed, and devenv
eventually printed Tests passed :) and returned zero. Ota therefore reported the selected task as
successful.

That is faithful to the current contract: Ota invokes the selected orchestrator operation and
uses its terminal result. It does not independently reinterpret every nested devenv process log.
The matrix consequently proves that Ota executed the declared boundary, not that every internal
devenv process completed successfully.

The task also modified four tracked files:

• devenv.lock
• frontend/elm-srcs.nix
• frontend/generated-api/.openapi-generator/VERSION
• frontend/generated-api/README.md

Those mutations are part of the observed repository behavior. The run did not assert that the
worktree remained clean, so the note must not turn its green status into a non-mutation claim.

Uncovered material behavior

The pressure result classifies the remaining behavior explicitly.

Contract-owned and proved

• native contract validation on Ubuntu and macOS
• task, workflow, and execution-topology discovery
• Doctor and Doctor fix-preview output, including the retained not_ready agent verdict
• native dry-run admission for test, verify, and dev
• real Ubuntu execution of the selected test task through the pinned Nix/devenv launcher

Explicitly bounded as not proved

• real task execution on macOS
• successful completion of every nested devenv process
• lifecycle readiness and teardown assertions for dev
• container and Windows execution
• deployment, production readiness, and repository-global governance
• a clean worktree after verification

Repo-owned outside this pressure slice

• the internal devenv process graph and its terminal acceptance semantics
• whether generated files should be committed, ignored, or checked for drift
• the missing cloud.devenv.toml configuration expected by backend-migrate

Ota platform gap

Ota does not currently receive structured nested-process outcomes from the devenv adapter. It can
govern the selected orchestrator operation and report its exit status, but it cannot prove that all
material child processes succeeded when the orchestrator itself returns zero. A stronger future
carrier would need typed nested outcome evidence rather than log scraping.

Why this pressure case matters

The result is useful precisely because it is not a staged perfect success.

Ota correctly preserved the Nix/devenv ownership boundary and executed the selected finite lane.
The same run then showed where that evidence stops. That distinction is the product: a green outer
command should never silently expand into a claim about nested work Ota did not independently
witness.

Links

• Immutable contract
• Immutable matrix workflow
• Hosted matrix run
• Ubuntu execution job
• macOS admission job

Originally Posted: https://ota.run/blog/pressure-testing-ota-on-cloud-devenv-sh-4q1n