Pressure-testing Ota on cloud.devenv.sh with Nix and devenv
A hosted Linux and macOS matrix tested how Ota models Nix as launcher and devenv as orchestrator, while keeping execution and mutation claims bounded.
Overview
cloud.devenv.sh is a useful Ota pressure case because its execution path crosses three distinct ownership boundaries:
- the host provides
nix - Nix launches a pinned devenv revision
- devenv owns the repository's
testandupsubcommands
A weak contract would flatten that into an opaque shell command. The stronger model preserves who owns each layer and lets Ota select the repository operation without pretending to own devenv's internal process semantics.
The immutable hosted matrix at repository revision 29c44c235795d3af3b1b7eafdaacd63a416d1f06 completed successfully on Ubuntu and macOS with released Ota v1.6.26. The result proved the modeled boundary, but it also exposed an important limit: a green orchestrator exit is not proof that every nested process succeeded.
Evidence snapshot
The hosted workflow run used:
- cloud.devenv.sh revision
29c44c235795d3af3b1b7eafdaacd63a416d1f06 - Ota
v1.6.26source revision08855fd1b7269abe9342dc9dd45acf4ed1885063 - contract-pinned devenv launcher revision
65a7c40d185350f0e96783b3dd8d4afab3bb7034 - Ubuntu binary SHA-256
3d545be8eb9701542284d28bb89fba83c77ae30c1d1157d9dd87fb83d961e05e - macOS binary SHA-256
a2448590b0fedfeaffd29535103d5209856e3b0fcb42a93d0f47256920b58a79
Both jobs validated the contract with zero errors and warnings, inspected Doctor's deterministic fix preview, listed the public task surface, exported execution topology, and dry-ran test, verify, dev, and the verify workflow. The contract intentionally declares no agent-safe task, and the retained output confirmed that ota tasks --safe --json returned an empty list.
Doctor did not call the repository ready. Its retained fix preview reported verdict: risky and agent_verdict: not_ready because .ota/state/, .ota/contracts/, .ota/receipts/, and .ota/proof/ were not ignored by Git. The workflow intentionally accepted that advisory posture; its green status must not be read as an agent-readiness verdict.
The Ubuntu job additionally executed the finite test task through Ota. The macOS job did not run that real task; it proved only validation, discovery, topology, Doctor, and dry-run admission.
The contract boundary
The contract keeps launcher ownership explicit:
orchestrators: devenv: kind: devenv required: true config_files: - devenv.nix launcher: exe: nix args: - run - github:cachix/devenv/65a7c40d185350f0e96783b3dd8d4afab3bb7034#devenv - --The repository's verification command remains a direct devenv subcommand:
tasks: test: command: exe: test execution: orchestrator: ref: devenv mode: subcommandThe long-running development path uses the same owner without being mislabeled as finite work:
tasks: dev: launch: kind: command exe: up execution: orchestrator: ref: devenv mode: subcommandThis separation gives Ota enough structure to explain and select the path while leaving devenv responsible for the process graph it launches.
What the hosted matrix proved
The retained artifacts and logs prove that:
- released Ota
v1.6.26loaded and validated the immutable contract on Ubuntu and macOS - Ota resolved
nixas the host requirement and the pinned devenv invocation as orchestrator truth - task discovery preserved finite
test, aggregateverify, and long-runningdevas distinct lanes - Doctor preserved the repository-hygiene warning and
not_readyagent verdict - all advertised native lanes admitted dry-run on both operating systems
- the Ubuntu job executed
ota run test --native --streamand Ota reported success after devenv
returned exit code zero
- the Ubuntu run stopped the devenv-managed processes before Ota emitted its successful run summary
This is execution evidence for one selected Ubuntu task. It is not runtime proof, lifecycle proof, or evidence that every repository behavior was governed.
The green run exposed a real boundary
During the Ubuntu task, devenv logged that backend-migrate failed because cloud.devenv.toml was missing. Other work continued, the frontend build completed, and devenv eventually printed Tests passed :) and returned zero. Ota therefore reported the selected task as successful.
That is faithful to the current contract: Ota invokes the selected orchestrator operation and uses its terminal result. It does not independently reinterpret every nested devenv process log. The matrix consequently proves that Ota executed the declared boundary, not that every internal devenv process completed successfully.
The task also modified four tracked files:
devenv.lockfrontend/elm-srcs.nixfrontend/generated-api/.openapi-generator/VERSIONfrontend/generated-api/README.md
Those mutations are part of the observed repository behavior. The run did not assert that the worktree remained clean, so the note must not turn its green status into a non-mutation claim.
Uncovered material behavior
The pressure result classifies the remaining behavior explicitly.
Contract-owned and proved
- native contract validation on Ubuntu and macOS
- task, workflow, and execution-topology discovery
- Doctor and Doctor fix-preview output, including the retained
not_readyagent verdict - native dry-run admission for
test,verify, anddev - real Ubuntu execution of the selected
testtask through the pinned Nix/devenv launcher
Explicitly bounded as not proved
- real task execution on macOS
- successful completion of every nested devenv process
- lifecycle readiness and teardown assertions for
dev - container and Windows execution
- deployment, production readiness, and repository-global governance
- a clean worktree after verification
Repo-owned outside this pressure slice
- the internal devenv process graph and its terminal acceptance semantics
- whether generated files should be committed, ignored, or checked for drift
- the missing
cloud.devenv.tomlconfiguration expected bybackend-migrate
Ota platform gap
Ota does not currently receive structured nested-process outcomes from the devenv adapter. It can govern the selected orchestrator operation and report its exit status, but it cannot prove that all material child processes succeeded when the orchestrator itself returns zero. A stronger future carrier would need typed nested outcome evidence rather than log scraping.
Why this pressure case matters
The result is useful precisely because it is not a staged perfect success.
Ota correctly preserved the Nix/devenv ownership boundary and executed the selected finite lane. The same run then showed where that evidence stops. That distinction is the product: a green outer command should never silently expand into a claim about nested work Ota did not independently witness.
Links
Take action