Pressure-testing Ota on Hasura: raw Kubernetes manifests and honest kubectl proof

Hasura exposed a useful execution-governance boundary: raw Kubernetes manifests without Helm or Compose owning the operational path.

That sounds simple. It is also easy to overclaim.

kubectl can parse and render a manifest locally without contacting a cluster. A Kubernetes API can accept those objects without proving that Hasura became ready. A green command can therefore mean at least three different things:

• the manifests were interpreted locally;
• a cluster accepted the declared resources;
• the application started and became usable.

This pressure test asked whether Ota could govern the first two claims while refusing to imply the third. The contract now installs released Ota v1.6.24 directly, so the version used by humans, agents, and CI is contract truth rather than workflow-local bootstrap glue.

The work stayed deliberately scoped to install-manifests/kubernetes. That directory contains the raw Hasura deployment and service manifests used by this test.

The Ota contract owns:

• released Ota v1.6.24 bootstrap identity;
• cross-platform kubectl acquisition;
• non-mutating local manifest tasks;
• explicit cluster-mutating apply tasks;
• a safe verification workflow;
• a separate deployment workflow;
• task and workflow dry-run coverage;
• the task and workflow truth consumed by the three-OS pressure matrix.

It did not attempt to govern all of Hasura or turn this narrow infrastructure directory into an application-readiness contract.

Ota owns its own bootstrap identity

The pressure contract does not ask CI to choose an implementation branch or duplicate installer commands. It pins the released version through the contract:

agent:  bootstrap:    ota:      source:        kind: version        version: v1.6.24

The GitHub workflow uses ota-run/setup with source: contract, so the workflow consumes this identity instead of becoming another source of Ota version truth.

Ota owns the tool

The contract does not hide kubectl installation in a shell preamble:

tools:  kubectl:    version: "*"    acquisition:      provider: apt      package: kubectl    platforms:      macos:        acquisition:          provider: brew          package: kubernetes-cli      windows:        acquisition:          provider: winget          package: Kubernetes.kubectl

That gives humans, CI, and agents the same declared tool requirement instead of three separate installation conventions.

Local verification is non-mutating

The safe verification lane uses kubectl annotate --local. It asks kubectl to load and emit the object without contacting a Kubernetes API:

k8s:preview:deployment:  command:    exe: kubectl    args:      - annotate      - --local      - -f      - deployment.yaml      - ota.run/preview=true      - -o      - yaml

The aggregate keeps both manifests in one canonical verification path:

verify:  description: Render the focused raw Kubernetes manifest set without mutating the cluster  aggregate:    tasks:      - k8s:preview:deployment      - k8s:preview:service

This proves that the installed kubectl can interpret and render the selected manifests. It does not prove server-side admission, policy compatibility, resource creation, or application readiness.

Cluster mutation is not agent-safe verification

The apply tasks declare their external effect instead of disguising deployment as another check:

k8s:apply:service:  command:    exe: kubectl    args:      - apply      - -f      - svc.yaml  effects:    external_state:      - kubernetes

Those tasks are intentionally excluded from agent.safe_tasks. The workflow layer also gives verification and deployment different operational meanings:

workflows:  default: verify verify:  intent: ci_verification  run:    task: verify deploy:  intent: deployment  run:    task: k8s:cluster:proof

Ota is not merely storing two commands. It is declaring which path is routine verification and which path mutates an external system.

The pressure design keeps three distinct proof levels:

This distinction is the main engineering result. A created Deployment and Service are stronger evidence than local YAML handling, but they are still not proof that Hasura became ready.

The upgraded contract was exercised with released Ota v1.6.24, commit c1ab23a58.

The current local proof covers:

• ota validate;
• ota doctor;
• ota tasks --use;
• ota tasks --safe --use;
• ota run verify --dry-run --json;
• ota up --workflow verify --dry-run --json;
• real ota run verify --native --agent;
• both real kubectl annotate --local tasks.

ota doctor reported the repository slice ready with no findings. The task and workflow previews were runnable, and the real safe verification closure completed successfully.

The pressure workflow is prepared to repeat contract discovery and dry-run coverage on Ubuntu, macOS, and Windows, with real kind cluster apply-and-observe proof on Ubuntu. That hosted matrix must pass against the upgraded contract before this draft is published as cross-platform v1.6.24 evidence.

Matrix design

The matrix is asymmetric by design. It does not pretend that every operating system owns a Kubernetes runtime:

The Ubuntu lane creates the kind cluster in GitHub Actions, outside the Ota contract. Ota then owns the declared kubectl task and workflow execution inside that provider-owned boundary. This keeps cluster provisioning truth separate from manifest execution truth.

The refreshed matrix acceptance bar is:

1. ota-run/setup installs the contract-pinned v1.6.24 release.
2. ota --version confirms the installed release before any contract claim is evaluated.
3. ota doctor passes without a tolerated blocked state.
4. The safe verify closure executes through the real agent boundary.
5. Successful JSON, topology, and execution-receipt evidence is retained for every lane.
6. Ubuntu proves resource apply and observation without promoting that result to application

readiness.

The pressure result remains narrow:

• Contract-owned and currently proved with v1.6.24: Ota bootstrap identity, kubectl acquisition shape, task/workflow selection, safe-task separation, local validation and discovery, dry-run admission, and real local manifest execution on macOS.
• Contract-owned and awaiting hosted reproof: Ubuntu kind resource apply/observation and the Ubuntu, macOS, and Windows matrix.
• Repo-owned and outside this slice: editing deployment.yaml with the real database URL.
• Not proved: Hasura pod readiness, database connectivity, service reachability, application behavior, production-cluster policy, RBAC, storage, ingress, load balancers, and rollback.
• Not governed by this contract: creation and teardown of the CI kind cluster, which remained GitHub workflow infrastructure.
• Not repo-global: no result here should be read as proof of the wider Hasura repository.

Raw manifests do not need a giant abstraction. They need precise ownership and honest evidence.

Ota adds value here by keeping four truths together:

• how the required tool is acquired;
• which command path is safe verification;
• which path mutates Kubernetes;
• what each successful lane actually proved.

The important outcome is not that Ota can run kubectl. Any shell can do that. The outcome is that humans, CI, and agents receive the same governed execution path without being told that local rendering, cluster acceptance, and application readiness are interchangeable.