Ready

mirendev/runtime#1413

62d49e3 fixes the authorization gap I raised last round. Rebuild now calls s.inner.authorizeBuild(ctx, name) right after AllowApp (rebuild.go:72-74), before it reads the app, the version, or the archive, so all three build entry points check authorization the same way. TestRebuildAuthorizesBeforeReturningAppState checks the right things:

I've resolved that thread.

The base-image refresh policy question (buildkit.go:157) is still open. The new Rebuild input only changes how build-arg env vars are computed. dockerfileFrontendAttrs() still sets force-pull for every Dockerfile build, so ordinary deploys refresh mutable FROM tags too. That thread was handed to a human, so I'm leaving it open for them to resolve.

I'm still not recommending approval. The change touches every deploy path, and whether to accept the registry-availability tradeoff behind that policy is the owner's call, not mine.

Replies to existing threads

Posted to the PR's comment threads when you submit.


Verdict: caveats

Data

{
  "owner": "mirendev",
  "repo": "runtime",
  "number": 1413,
  "verdict": "caveats",
  "event": "comment",
  "summary": "62d49e3 fixes the authorization gap I raised last round. `Rebuild` now calls `s.inner.authorizeBuild(ctx, name)` right after `AllowApp` (`rebuild.go:72-74`), before it reads the app, the version, or the archive, so all three build entry points check authorization the same way. `TestRebuildAuthorizesBeforeReturningAppState` checks the right things:\n- A denied caller gets `ErrUnauthorized` both for an app that exists and for one that doesn't, so the error can't be used to tell which apps exist.\n- An authorized caller still reaches the \"no deployed version\" check, so the gate isn't simply rejecting everyone.\n\nI've resolved that thread.\n\nThe base-image refresh policy question (`buildkit.go:157`) is still open. The new `Rebuild` input only changes how build-arg env vars are computed. `dockerfileFrontendAttrs()` still sets force-pull for every Dockerfile build, so ordinary deploys refresh mutable `FROM` tags too. That thread was handed to a human, so I'm leaving it open for them to resolve.\n\nI'm still not recommending approval. The change touches every deploy path, and whether to accept the registry-availability tradeoff behind that policy is the owner's call, not mine.",
  "replies": [
    {
      "thread_id": "PRRT_kwDONZ__ss6wig67",
      "body": "Checked 62d49e3. `authorizeBuild` now runs right after `AllowApp`, before the app and version lookups and before the archive is opened. The new test checks what matters. A denied caller gets `ErrUnauthorized` whether or not the app exists, so the error can't be used to probe for apps. The authorized control still reaches app-state validation, which shows the gate isn't blanket-rejecting. That's the fix I asked for.",
      "resolve": true,
      "resolve_reason": "addressed"
    }
  ],
  "posted_to_pr": true,
  "draft": true,
  "head_sha": "62d49e3ebd8247dd14ededaadf25a28e7e6ab9e3"
}
Previous revisions (3)
Revision 3 · 20d ago

mirendev/runtime#1413

Since my last review the branch has been rebased onto main, and GitHub now reports it as mergeable, so the conflict blocker is gone. The retention fix came through unchanged as 8ab3005. receiveTar still warns and continues when retainSource fails, and every error path still returns an empty digest.

Still open: base-image refresh policy. Someone still has to decide whether every Dockerfile build should force-pull base images or only rebuilds should (buildkit.go:157). dockerfileFrontendAttrs() still sets image-resolve-mode=pull for both Transform and buildImage. I'm leaving that thread open for the owner.

New: Rebuild reads app state before it checks authorization. BuildFromTar and BuildFromPrepared call authorizeBuild right after AllowApp. SagaBuilder.Rebuild (rebuild.go:66-96) doesn't. Under logical authorization, the app-level deploy check only happens later, when startBuild calls ensureApp (build.go:1207-1208).

The saga never starts for an unauthorized caller, so nothing gets deployed. But before that check, Rebuild has already:

  • looked up the app and its active version,
  • opened the retained archive,
  • returned errors that differ depending on the app's state ("no deployed version", "not built from stackbuild or a Dockerfile", "no retained source", "deploy again").

A caller who is allowed to call builder.rebuild but has no deploy permission on an app can use those errors to learn whether the app exists and how it was deployed. It's low severity, and it's a one-line fix: call s.inner.authorizeBuild(ctx, name) at the top, as the other two entry points do.

I'm not recommending approval. The change touches every deploy path, the base-image policy needs a human decision, and new RPC entry points that handle authorization deserve a person's eye.

Inline comments

  • servers/build/rebuild.go:69 — BuildFromTar and BuildFromPrepared call s.inner.authorizeBuild(ctx, name) here, before doing anything else. Rebuild only checks AllowApp. Under logical authorization, the app-level deploy check doesn't run until startBuild → ensureApp, after this handler has already read the app and version, opened the archive, and possibly returned one of four state-specific errors. Nothing gets deployed without authorization. But a caller with builder.rebuild and no deploy permission on the app can tell from those errors whether the app exists and how it was built. I'd call authorizeBuild right after AllowApp so all three entry points enforce authorization the same way.

Verdict: caveats

Revision 2 · 20d ago

mirendev/runtime#1413

The retention fix in 86aae6d does what it set out to do. When retainSource fails, receiveTar now logs a warning and continues. Every error return in retainSource (rebuild.go) gives back an empty digest, so a failed write leaves SourceArchive empty instead of recording a digest that has no file behind it. The deferred os.Remove cleans up a partly written temp file if the disk fills mid-write. TestSourceRetentionFailureDoesNotBlockDeploy blocks writes by putting a file where the build-sources directory should be, which fails even when the test runs as root. The test covers both normal deploys and previews.

What happens downstream when retention fails is reasonable:

  • If retention fails during a rebuild, the new version has no archive, so the next rebuild tells the operator to deploy again.
  • PrepareUpload already falls back to a full upload when stageMatchingFiles errors or there's no archive (build.go:1399-1415).

I'm resolving that thread. You've already acknowledged that there's still no archive GC.

The force-pull of Dockerfile base images (buildkit.go:157) is still open. You're leaving the choice between refreshing on every build and refreshing only on rebuild to the owner. That's the right call, and I've left the thread open for whoever makes it.

GitHub still reports the branch as not mergeable, so it needs a rebase. I'm not recommending approval: the change touches every deploy path, the base-image question is still unanswered, and the branch has conflicts.

Replies to existing threads

Posted to the PR's comment threads when you submit.

  • (reply & resolve: addressed) I checked 86aae6d. receiveTar now logs a warning and continues, and every error path in retainSource returns an empty digest. A failed write therefore can't leave a version pointing at an archive that doesn't exist, and the deferred os.Remove cleans up a partly written temp file. Putting a file where the build-sources directory should be is a good way to test this, because it fails even when the test runs as root. The test covers both deploys and previews and asserts they succeed with an empty SourceArchive. This is the fix I asked for. Agreed that GC is a separate limitation.
  • (reply) Thanks for confirming both entry points. I'm leaving this open until the owner decides whether every build refreshes base images or only rebuilds do. If they choose rebuild-only, gating the attr on runBuildkitBuildInputs.Rebuild should be a small change.

Verdict: caveats

Revision 1 · 20d ago

mirendev/runtime#1413

I think the rebuild path is sound and this can go to human review. Two behavior changes reach every deploy, not just miren app rebuild, and someone should accept them on purpose before this merges. GitHub also reports the branch as not mergeable right now, so it needs a rebase either way.

What holds up

  • Rebuild checks rpc.AllowApp.
  • It refuses image-only, static and legacy versions with useful errors.
  • It re-checks that the version it picked is still active, and it does that inside loadSource, after the deployment lock is taken (load-source now depends on begin-deployment). TestRebuildRejectsChangedActiveVersion confirms the lock is released when that check fails.
  • On a rebuild, setActiveVersion skips applyWorkloadRole, so an operator-set role isn't overwritten by the retained app.toml.
  • Making archives content-addressed and immutable let you delete all of the appLocks / manifest code from source_cache.go.
  • Delta uploads read from the active version's archive and check each file's hash before reusing it. A preview therefore can't change what the next normal deploy diffs against, and a deploy or rollback between PrepareUpload and BuildFromPrepared can't put the wrong bytes into a session.

Caveats to accept or fix

  1. Retention now fails the deploy, and nothing ever deletes archives (build_saga.go:97-100). The old source cache was best-effort: a failed write only logged a warning. Now every deploy, preview and rebuild writes a full source .tar.gz under DataPath/build-sources, with no garbage collection, and any write failure fails the build. Over time the data disk fills, and once it does, every deploy on the cluster fails, not just rebuilds. The description mentions the missing GC, but not that the two together turn disk growth into a deploy outage. I'd make retention best-effort (leave SourceArchive empty; Rebuild and stageMatchingFiles already handle that), or ship at least basic cleanup alongside this.
  2. Every Dockerfile build now force-pulls its base image (buildkit.go:157). image-resolve-mode: pull is set in dockerfileFrontendAttrs(), which normal miren deploy uses too, not only rebuilds. Clusters with no or flaky registry access, or that run into Docker Hub limits, will start seeing deploy failures where the cached base image used to work. The description calls this out. A human should decide whether it applies to all builds or only to rebuild.

One deliberate behavior worth knowing: rebuilds keep the existing ConfigSpec as-is (prepareConfig, spec := existing). If a refreshed base image changes WORKDIR or EXPOSE, the new version keeps the old start directory and port. That matches the stated goal of preserving runtime config, so I'm only noting it.

I'm not recommending approval: this is a large change that touches every deploy path, and it currently has merge conflicts.

Inline comments

  • servers/build/build_saga.go:97 — Before this change, a failed source-cache write only logged a warning. Now a retainSource failure fails the build, for normal deploys and ephemeral previews as well as rebuilds. Archives are also never deleted, so build-sources grows with every deploy. Once the data disk fills, every deploy on the cluster fails here. Could retention be best-effort, logging a warning and returning an empty SourceArchive? Rebuild already gives a clear "deploy again" error when the archive is missing, and stageMatchingFiles already falls back to a full upload, so a failed write would degrade only rebuild and delta upload, not deploys. The other way to keep this fatal is to ship cleanup in the same change.
  • servers/build/buildkit.go:157 — This needs a human decision. dockerfileFrontendAttrs() is used by every Dockerfile build (buildImage and Transform), so normal miren deploy now force-resolves FROM tags against the registry too, not only app rebuild. Clusters that can't reach the registry reliably (air-gapped, flaky networks, Docker Hub rate limits) would start failing deploys that used to succeed from cache. If the goal is just to let operators refresh base images on demand, it could be applied only when runBuildkitBuildInputs.Rebuild is true. If refreshing on every deploy is intended, the description already says so, but it's a cluster-wide behavior change that someone should explicitly accept.

Verdict: caveats