Image Tags That Don't Lie: CI-Owned Tags and Three Secret Gates for an Agent Image

The agent I run in a container is built from a repository that holds its persona, skills, scripts and config. That is exactly the kind of tree where a stray token can ride along into an image. Two problems came up as I automated the build, and they turned out to be the same problem seen from two angles: who is allowed to publish what, and what is allowed to go in.

This post covers the tagging scheme that stopped the registry lying to me, and the three gates that run before anything is baked.

(For what is in the container and how it behaves, see Packing an Agent into a Container.)

Part 1 — one owner per tag

There are two ways an image gets built and pushed: a CI workflow on GitHub, and a manual script I run on a lab host when CI isn’t an option. For a while both pushed :latest.

The result was predictable in hindsight. :latest silently became whichever pipeline ran last, so a hand-built image could roll back what CI had just published, and nothing in the registry showed it had happened.

The fix is to give every tag exactly one owner.

The manual script is explicitly forbidden from touching :latest:

# This is the MANUAL fallback. CI is the canonical publisher: it owns
# :latest, :<sha7>, :<run-number> and :YYYY.MM.DD.
#
# This script deliberately does NOT touch :latest. Both pipelines used to push
# it, so the tag silently flipped to whichever ran last.
echo "== push $IMAGE:$STAMP (not :latest — CI owns that tag)"
docker push "$IMAGE:$STAMP"

CI builds all four tags from one docker build, and stamps the image with the commit it came from:

DATE_TAG="$(date -u +%Y.%m.%d)"
docker build -f container/Dockerfile \
  --label org.opencontainers.image.revision=${{ github.sha }} \
  -t "$IMAGE:${GITHUB_SHA::7}" -t "$IMAGE:${GITHUB_RUN_NUMBER}" \
  -t "$IMAGE:$DATE_TAG" -t "$IMAGE:latest" .

The org.opencontainers.image.revision label means any running container can be traced back to its commit with docker inspect, even if someone only knows the tag it was pulled by.

A second, quieter bug: the stale base image

A related trap lives in the manual path. A local build uses whatever base image is already cached. On 2026-10-29 I found a locally cached base that was three months older than the one CI used, so a hand-built image carried an old runtime under a brand-new date tag. The tag said “fresh” and the contents were not.

The script now pulls the base before every build:

echo "== refresh base image"
docker pull nousresearch/hermes-agent:latest

A date in a tag describes when you built the image, not how current its ingredients are. If you want the tag to mean more, pin the base by digest.

Part 2 — three gates before anything is baked

The reason for the gates is one sentence in the build script: an image layer is permanent. If a credential gets copied into a layer and pushed, deleting it from a later commit does nothing; the layer still holds it. So the checks happen before the build, and they refuse to continue rather than warn.

Gate 1: pattern scan

tools/scan-repo.py scans every text file for credential formats and exits non-zero on a finding. The “hard” patterns include:

• private key headers (-----BEGIN … PRIVATE KEY-----)
• GitHub tokens (gho_, ghp_, ghu_, ghs_, ghr_)
• AWS access key IDs
• Slack tokens, JWTs, and several AI-provider key formats
• Ceph auth keys
• Telegram bot tokens
• an RKE2 join token, and passwords assigned in Proxmox-style variables

Beyond known formats, it also flags any password = …, secret = …, api_key = … or token = … assignment whose right-hand side is a literal instead of an environment lookup. Placeholder shapes like <token>, changeme, your_key and xxxx are allow-listed, and so is anything that expands from the environment ($VAR, os.environ, getenv).

Real false positives go in an allow-list file, but with a rule:

The rule name is matched; the line number is informational (files shift) — so an entry cannot silently start covering a different finding in the same file.

That small design choice is what keeps an allow-list from turning into a hole.

Gate 2: value cross-check

Pattern matching can’t find a secret that has no recognisable shape — a plain password that appears in prose rather than as password = … matches nothing. So gate 2 goes the other way round: it loads every credential-shaped value from the credential files on the host (the agent’s .env, a .env, cloud.conf, the Rancher and OpenStack openrc files, the Kolla passwords.yml) and searches the repo for each one as a literal. It reports only which file matched and which credential key the value came from. The values themselves are never printed.

A few extra known-bad values (a lab-wide default password, say) can be supplied through an environment variable, deliberately not hard-coded, because the script is itself committed.

The catch is that it needs those live files, so it cannot run in CI. The workflow says so openly:

# Gate 2 (value cross-check) needs the live host's credential files and cannot run
# here — it is mandatory on the host before the push that triggers this workflow.

The manual build script enforces it, aborting unless the cross-check reports clean:

echo "== gate 2: value cross-check"
if ! python3 tools/crosscheck-secrets.py | tail -2 | grep -q BERSIH; then
  echo "ABORT: gate 2 is not clean. Fix the finding before baking an image."; exit 2
fi

(BERSIH is Malay for “clean”.) The honest weakness: for the CI path, gate 2 depends on me running it on the host first. That is a process control, not a technical one.

Gate 3: no credential-shaped files in the build context

The simplest gate, and the one that never fails to catch a mistake: look at file names.

if find skills config -type f \( -name '*.keyring' -o -name '*.pem' \
     -o -name 'cloud.conf*' -o -name 'passwords.yml' -o -name 'kubeconfig*' \
     -o -name '*.env' \) | grep -q .; then
  echo "credential-shaped file present in skills/ or config/"; exit 2
fi

A kubeconfig, a Ceph keyring, a passwords.yml or a .env has no business in an image. Failing on the name is cheap and catches the case where the content scan is fooled.

The workflow around them

CI triggers on a push to master only when something that goes into the image changes (skills/**, config/**, container/**, the scanner itself), or on demand:

on:
  workflow_dispatch:
  push:
    branches: [master]
    paths: ['skills/**', 'config/**', 'container/**', 'tools/scan-repo.py']

permissions:
  contents: read
  packages: write

It publishes with the workflow’s own GITHUB_TOKEN. That token can push packages, so no personal access token with write:packages has to be created or handed around, and there’s nothing long-lived to leak.

After pushing, a summary line records what was published (### pushed :<sha7>, :<date> and :latest) in the run page, and a smoke test runs the freshly built image:

docker run --rm --entrypoint bash "$IMAGE:latest" -lc \
  'n=$(find /opt/data/skills -name SKILL.md | wc -l); echo "skills=$n"; test "$n" -ge 100'

One observation on my own pipeline: that step is named “skills present, no secrets dir”, but the command shown only checks the skill count. Adding an actual test ! -e /opt/data/secrets would make the step do what its name promises.

What I took from it

1. Every tag gets one owner. If two pipelines can write a tag, that tag is a race, and the registry will show you the winner without telling you there was a race.
2. Keep an immutable tag next to the moving one. :latest is for convenience; :<sha7> is for reproducibility and rollback.
3. Refresh the base before building. A fresh tag on a stale base is a lie of omission.
4. Gate before the build, not after. Layers are permanent, so scanning an image that’s already been pushed is too late.
5. Use more than one kind of check. Patterns catch known formats, value cross-checks catch the rest, and file-name rules catch the careless case. Each one has a blind spot the others cover.
6. Be honest about the control that is manual. Gate 2 can’t run in CI. Writing that in the workflow file is better than letting a green tick imply it ran.

The weak point of the whole design is that one gate relies on a human running it first. If I rebuilt it, I’d look for a way to make the value cross-check available to CI — for example by checking against a hashed list of known secret values rather than the secrets themselves.