← Back to blog
Engineering note2026-08-25 12:18 UTC

Pressure-testing Ota on cloud.devenv.sh with Nix and devenv

A hosted Linux and macOS matrix tested how Ota models Nix as launcher and devenv as orchestrator, while keeping execution and mutation claims bounded.

Overview

cloud.devenv.sh is a useful Ota pressure case because its execution path crosses three distinct ownership boundaries:

  • the host provides nix
  • Nix launches a pinned devenv revision
  • devenv owns the repository's test and up subcommands

A weak contract would flatten that into an opaque shell command. The stronger model preserves who owns each layer and lets Ota select the repository operation without pretending to own devenv's internal process semantics.

The immutable hosted matrix at repository revision 29c44c235795d3af3b1b7eafdaacd63a416d1f06 completed successfully on Ubuntu and macOS with released Ota v1.6.26. The result proved the modeled boundary, but it also exposed an important limit: a green orchestrator exit is not proof that every nested process succeeded.

Evidence snapshot

The hosted workflow run used:

Both jobs validated the contract with zero errors and warnings, inspected Doctor's deterministic fix preview, listed the public task surface, exported execution topology, and dry-ran test, verify, dev, and the verify workflow. The contract intentionally declares no agent-safe task, and the retained output confirmed that ota tasks --safe --json returned an empty list.

Doctor did not call the repository ready. Its retained fix preview reported verdict: risky and agent_verdict: not_ready because .ota/state/, .ota/contracts/, .ota/receipts/, and .ota/proof/ were not ignored by Git. The workflow intentionally accepted that advisory posture; its green status must not be read as an agent-readiness verdict.

The Ubuntu job additionally executed the finite test task through Ota. The macOS job did not run that real task; it proved only validation, discovery, topology, Doctor, and dry-run admission.

The contract boundary

The contract keeps launcher ownership explicit:

NIX-OWNED DEVENV LAUNCHERyaml
orchestrators:  devenv:    kind: devenv    required: true    config_files:      - devenv.nix    launcher:      exe: nix      args:        - run        - github:cachix/devenv/65a7c40d185350f0e96783b3dd8d4afab3bb7034#devenv        - --

The repository's verification command remains a direct devenv subcommand:

FINITE VERIFICATION TASKyaml
tasks:  test:    command:      exe: test    execution:      orchestrator:        ref: devenv        mode: subcommand

The long-running development path uses the same owner without being mislabeled as finite work:

LONG-RUNNING DEVELOPMENT TASKyaml
tasks:  dev:    launch:      kind: command      exe: up    execution:      orchestrator:        ref: devenv        mode: subcommand

This separation gives Ota enough structure to explain and select the path while leaving devenv responsible for the process graph it launches.

What the hosted matrix proved

The retained artifacts and logs prove that:

  • released Ota v1.6.26 loaded and validated the immutable contract on Ubuntu and macOS
  • Ota resolved nix as the host requirement and the pinned devenv invocation as orchestrator truth
  • task discovery preserved finite test, aggregate verify, and long-running dev as distinct lanes
  • Doctor preserved the repository-hygiene warning and not_ready agent verdict
  • all advertised native lanes admitted dry-run on both operating systems
  • the Ubuntu job executed ota run test --native --stream and Ota reported success after devenv

returned exit code zero

  • the Ubuntu run stopped the devenv-managed processes before Ota emitted its successful run summary

This is execution evidence for one selected Ubuntu task. It is not runtime proof, lifecycle proof, or evidence that every repository behavior was governed.

The green run exposed a real boundary

During the Ubuntu task, devenv logged that backend-migrate failed because cloud.devenv.toml was missing. Other work continued, the frontend build completed, and devenv eventually printed Tests passed :) and returned zero. Ota therefore reported the selected task as successful.

That is faithful to the current contract: Ota invokes the selected orchestrator operation and uses its terminal result. It does not independently reinterpret every nested devenv process log. The matrix consequently proves that Ota executed the declared boundary, not that every internal devenv process completed successfully.

The task also modified four tracked files:

  • devenv.lock
  • frontend/elm-srcs.nix
  • frontend/generated-api/.openapi-generator/VERSION
  • frontend/generated-api/README.md

Those mutations are part of the observed repository behavior. The run did not assert that the worktree remained clean, so the note must not turn its green status into a non-mutation claim.

Uncovered material behavior

The pressure result classifies the remaining behavior explicitly.

Contract-owned and proved

  • native contract validation on Ubuntu and macOS
  • task, workflow, and execution-topology discovery
  • Doctor and Doctor fix-preview output, including the retained not_ready agent verdict
  • native dry-run admission for test, verify, and dev
  • real Ubuntu execution of the selected test task through the pinned Nix/devenv launcher

Explicitly bounded as not proved

  • real task execution on macOS
  • successful completion of every nested devenv process
  • lifecycle readiness and teardown assertions for dev
  • container and Windows execution
  • deployment, production readiness, and repository-global governance
  • a clean worktree after verification

Repo-owned outside this pressure slice

  • the internal devenv process graph and its terminal acceptance semantics
  • whether generated files should be committed, ignored, or checked for drift
  • the missing cloud.devenv.toml configuration expected by backend-migrate

Ota platform gap

Ota does not currently receive structured nested-process outcomes from the devenv adapter. It can govern the selected orchestrator operation and report its exit status, but it cannot prove that all material child processes succeeded when the orchestrator itself returns zero. A stronger future carrier would need typed nested outcome evidence rather than log scraping.

Why this pressure case matters

The result is useful precisely because it is not a staged perfect success.

Ota correctly preserved the Nix/devenv ownership boundary and executed the selected finite lane. The same run then showed where that evidence stops. That distinction is the product: a green outer command should never silently expand into a claim about nested work Ota did not independently witness.