On this page
Every infrastructure-as-code tutorial I read had the same opening move: start with an empty cloud account, write some code, watch a server appear. Mine was not an empty account. Mine was four machines in a cupboard running ten containers that my house genuinely depends on — DNS, backups, the wiki, a game server the kids play on — all of it built by hand over months, none of it described anywhere except in my own head.
The gap between those two situations is the whole problem. Following the standard advice on a live lab means letting the tool create fresh copies of things you already have, which is a polite way of saying “delete your homelab and rebuild it.” I wasn’t going to do that. So I took the other route, the one the tutorials skip: describe what already exists, prove the description is exact, and never let the tool write anything at all.
This is part one of a series about codifying a running Proxmox cluster. It’s the part about posture — what you’re actually trying to achieve, what code can and can’t save you from, and how to set things up so a mistake is impossible rather than merely unlikely. Part two is the hands-on import recipe.
Why “build it from scratch” is the wrong goal
The appeal of infrastructure as code is usually pitched as reproducibility: delete everything, run one command, get it all back. That’s a genuine superpower when your infrastructure is disposable — when a server is a thing you rent for an afternoon.
A homelab is not disposable. The container running Pi-hole has months of configuration in it. The wiki container has the notes I’d need to rebuild everything else. “Destroy and recreate” isn’t a recovery plan for those, it’s the disaster.
So the goal changes. Instead of code that can build my lab, I wanted code that describes my lab exactly — and the value comes from somewhere less obvious than rebuilding.
Once your code matches reality exactly, any difference between the two is a change someone made and didn’t write down. A single command tells you whether your lab still looks the way you think it does. That drift detector turns out to be worth more, week to week, than the ability to recreate everything from nothing.
There’s a second payoff, which is that writing the description forces you to actually look. When I codified my four nodes, I discovered every one of them was running a monitoring agent from a manually-downloaded binary while the system package sat installed-but-inert alongside it. Nothing was broken, and my own documentation said something different. Writing the code corrected the documentation — a pattern that repeated several times before I was finished.
What code can bring back, and what it can’t
This is the part I’d want someone to tell me before I started, because getting it wrong produces false confidence — the most expensive kind.
Infrastructure code describes the shape of a container: how much memory it gets, how big its disk is, which node it lives on, what its network settings are, whether it starts on boot. Recreate a container from that description and you get a correctly-shaped empty box.
What made it your container — the installed packages, the databases, the config files, the media library, the years of accumulated state — was never in the code. That lives in your backups.
The honest framing is that code gets you a correctly-arranged empty lab very quickly, and that’s genuinely useful — recreating ten containers with the right settings by hand is an afternoon of tedious clicking and at least one typo. But it’s the first half of a recovery, not the whole thing. Anyone who tells you their infrastructure is “fully in code” and doesn’t mention backups has described half a system.
The same split applies to the hosts. Code can reapply the things you configured on top of a fresh Proxmox install — the monitoring agents, the mesh networking, the scheduled scripts. It cannot reinstall Proxmox itself, and it cannot bring back the API tokens, because those live in a cluster database that dies with the machine. Every token has to be reissued by hand after a rebuild. That’s worth writing down before you need it.
Give the tool a credential that cannot break anything
Here’s the decision that makes all of this safe, and it’s the one I’d argue hardest for.
Proxmox VE has a built-in role called PVEAuditor, described in the official documentation as having read only access. Create an API token, give it that role and nothing else, and hand that to your infrastructure tool.
The token can read every setting it needs to build an accurate description of your cluster. It cannot create, modify, or delete a single guest. Not because you configured the tool carefully — because the API will refuse.
Everything below uses stand-in values. 10.0.0.73 is a documentation address — use your own node’s address. terraform@pve and tf are the user and token names I picked; use whatever names you like. pvelab01 through pvelab04 are my node names and yours will differ. Container IDs like 100 are examples — use the IDs your cluster actually assigned. The token secret Proxmox shows you goes into your own secret store, never into a file you commit. If a value looks specific to one machine, it’s a placeholder to change — not a literal to copy.
# Create a dedicated user for the tool
pveum user add terraform@pve
# Grant read-only access at the root of the permission tree
pveum acl modify / --users terraform@pve --roles PVEAuditor
# Create the token. Copy the secret it prints — it is shown once.
pveum user token add terraform@pve tf --privsep 1
The --privsep 1 flag is worth understanding rather than copying — and it’s the documented default, so writing it out is me being explicit about a choice, not switching something on. Proxmox calls it privilege separation, and with it enabled the token’s effective permissions are, in the documentation’s words, “calculated by intersecting user and token permissions.” Two locks in series: the token can only do what the user is allowed to do and what the token is separately granted. Since the user is an auditor, the ceiling is read-only no matter what happens to the token’s own grants later.
Because permissions are an intersection, a privilege-separated token can never exceed its user. If you later need a token that can write — to build a throwaway test container, say — do not promote the auditor user to get it. Create a separate short-lived user for that job and delete it afterwards. Keeping one identity permanently read-only is the whole point; widening it “just for a minute” quietly removes the safety net from everything else that uses it.
Why prevent_destroy is the second lock, not the first
Both OpenTofu — the open source fork created after HashiCorp relicensed Terraform under the Business Source License — and Terraform itself offer a lifecycle setting called prevent_destroy. Set it on a resource and the tool will refuse to proceed with any plan that would destroy that object. It looks like exactly the guardrail you want, and I do set it on every imported container.
But read the documentation carefully, because there’s a gap in it that surprised me:
“Since this argument must be present in configuration for the protection to be active, this setting cannot prevent the remote object from being destroyed when the
resourceblock is removed from configuration entirely: in that case, theprevent_destroysetting is removed along with it, and so OpenTofu would allow the destroy operation to succeed.”
The protection lives inside the thing it’s protecting. Delete the block — a careless edit, a bad merge, a tidy-up of a file you thought was unused — and you delete the guard along with it. The tool will then cheerfully destroy the container, and it will be right to, because as far as the configuration is concerned you asked for that.
This is why I keep insisting on the read-only token. prevent_destroy protects against the plan you didn’t read properly. The auditor token protects against everything else, including the edit you make six months from now in a repo you’ve half-forgotten, because it’s enforced on the server where no change to your files can reach it.
Belt and braces, in that order of importance.
The posture, in four rules
Everything above condenses to a short list. These are the rules I hold to across the rest of the series:
- Import, never create. The code describes what exists. Bringing a guest under management must not restart it, resize it, or touch it in any way.
- The credential is read-only, permanently. Write access is minted separately, scoped narrowly, used briefly, and deleted. It is never a property of the main identity.
prevent_destroyon everything imported. A second lock, understood as second.- An exact match is the deliverable. The artifact isn’t a lab you can rebuild — it’s a command that says “no changes” and therefore means something when it doesn’t.
That fourth one is the subject of the next post. Describing a running container accurately enough that a tool agrees, attribute for attribute, is harder than it sounds — the provider fills in defaults you never set, trims whitespace you didn’t notice, and generates a starter configuration with invalid values in it. Getting from “roughly right” to a genuinely clean plan is where the real work is, and it’s what part two walks through end to end.
You need a working cluster before any of this is worth doing. If you’re earlier in the journey, set up Proxmox first, then get your backups running — code is the second half of a recovery plan and backups are the first.
Related posts:
- Import a Live Proxmox Cluster Into OpenTofu With Zero Changes — part two: the hands-on import recipe that turns this posture into a clean plan
- Homelab Capacity Planning: What If a Node Dies Tonight? — the other half of thinking about failure, using measured RAM rather than configured limits
- Proxmox Backup Server: Automated CT and VM Backups with Deduplication — the contents half of recovery, which code cannot replace
- Forming a Proxmox Cluster: Quorum, Corosync, and Joining Multiple Nodes — the multi-node setup this series assumes you already have
- Proxmox Security Hardening: SSH Keys, Firewall, and Locking Down a New Cluster — where API tokens and permission scoping fit into the wider access model
- Never Commit a Secret to Your Knowledge Base — where the token secret goes instead of into your repo
- Creating Your First Proxmox LXC Container: Step-by-Step — what the imported resources are actually describing
Comments
Comments are powered by GitHub Discussions — sign in with a GitHub account to join the conversation.