On this page
There’s a tempting shortcut for keeping a knowledge base “always in sync”: a cron job that commits and pushes everything, everywhere, on a timer. Do not do this to your canonical vault. This is a ⚠️ post because blanket sync is the single fastest way to corrupt the one store that’s supposed to be always-right.
Writes to the authority go through a controlled-write contract, never a blanket sync: lock → validate → commit → refresh projections → verify → unlock. Any failure aborts the write and releases the lock, leaving the last good state intact. One writer at a time; every change atomic.
Adapt the specifics to your setup: the script and lock paths (/tmp/my-write.sh, /wiki-data/.canonical-writer.lock, canonical-writer-lock.py) are examples — point them at your own repo and tooling. The sequence is what matters, not the exact paths.
Why blanket sync corrupts
An auto-sync job has no opinion about what it’s pushing. It publishes broken links, committed secrets, and half-finished edits with equal enthusiasm — and then every projection faithfully rebuilds from that bad state. Worse, two sync jobs (your laptop and a cluster agent) can race and interleave commits.
The contract, step by step
In my lab the whole sequence is a documented runbook — a canonical write procedure — wrapped so the actual edit runs under the lock as a script.
A file lock (flock) serializes writers. The write is wrapped so the edit only runs while the lock is held, and the lock is always released — even on abort.
# The edit runs as a /tmp script under the lock — never inline
canonical-writer-lock.py -- bash /tmp/my-write.sh
Pre-flight audits reject broken links, stale paths, missing frontmatter, and — critically — anything secret-shaped. If validation fails, nothing is committed.
The link/secret audit reads the file list from standard input, not an argument: pipe it in with git ls-files -s | node scripts/wiki-audit.mjs .. Invoked bare, an older version blocked forever waiting on stdin — a “hang” that was misdiagnosed for ages as a slow network mount. The mount was fine; the tool was waiting for input that never came. Build validation CLIs to fail fast with usage when they get no input, and always feed them explicitly.
git ls-files -s | node scripts/wiki-audit.mjs .
# link check · stale-path check · frontmatter check · secret scan
Only after validation passes: commit the enumerated files (never git add -A), then rebuild the projections and confirm each reports the new commit.
git add <enumerated files> # never 'git add -A' on the canonical vault
git commit -m "Describe the change"
# refresh projections (server-side), then verify each reports HEAD
refresh-projection && verify-projection
build-index && verify-index
The wrapper releases the lock in its cleanup path. If you must abort a wrapped write, send SIGINT (not SIGTERM) so the wrapper’s cleanup runs and the lock is freed rather than left held.
A lock that survives a crashed write blocks every future write. Ensure the abort signal triggers the release path, and have a way to confirm the lock is free afterwards. A stuck lock is its own outage.
The complete wrapper, the stdin-fed audit, and the SIGINT-safe abort are in the paired playbook: Canonical Writer Lock + Validation Gates.
Collapsing the procedure into three verbs
Once you’ve run the manual steps enough times to trust them, you can collapse the sequence into a thin orchestrator that automates the boilerplate while keeping every gate identical. In my lab, that’s vault.mjs — three commands that mirror the four manual steps above:
# 1. preflight: regenerates topic-map, runs all audits, emits the stage list
node tools/canonical-vault/vault.mjs preflight
# 2. write: lock → commit → refresh → mirror → verify (one command, one JSON result)
node tools/canonical-vault/vault.mjs write --message 'feat: ...' --stage '<stage_list>'
# 3. verify: independent confirmation — all consumers and mirrors at HEAD
node tools/canonical-vault/vault.mjs verify
If vault.mjs itself is broken or the environment is degraded, you still need to be able to execute each step by hand. Keep the manual procedure documented and drilled — the orchestrator saves time on the normal path; the manual path gets you out of a stuck state.
Why a documented procedure beats a clever one-liner
The write sequence lived, for a while, only in someone’s shell history — and got mis-run twice (the wrong stdin, nested quoting inside a wrapped bash -c). The fix wasn’t a smarter command; it was writing the procedure down as a runbook and running edits as a /tmp script under the lock, never as inline nested quoting. A canonical write is exactly the operation you want boring, explicit, and identical every time.
The swap-in box
Git hosting (GitHub/GitLab): a protected branch + required CI checks is a controlled-write contract — validation gates the merge, and one merge lands at a time. CI/CD generally: model it as lock (the pipeline is serialized) → validate (test stage) → commit/deploy → verify (smoke test). Databases: this is just a transaction with post-commit consistency checks. The pattern is universal: never mutate the authority without validating first and verifying after.
What you have now
Writes that are atomic, validated, serialized, and always leave projections coherent — the opposite of fire-and-forget sync. That safety is the foundation for letting automated writers touch the vault at all, which is where the series turns next: cold-starting your AI so agents read and write the brain under the same contract.
Related posts:
- Turn Your Git Vault Into a Searchable Wiki.js Site — the projections this contract keeps coherent
- Make Your AI Agents Read the Docs First, Every Time — agents that write through this same contract
- Prove Your Homelab Docs Are Actually Up to Date — verifying the refresh step actually landed
- Never Commit a Secret to Your Knowledge Base — what the validation gate’s secret scan enforces
- Already Made a Mess of Your Notes? Here’s the Fix — cleaning up after blanket sync did the damage
- 10 Mistakes I Made Building a Homelab Second Brain — the silent-success failures this post’s contract prevents
Comments
Comments are powered by GitHub Discussions — sign in with a GitHub account to join the conversation.