Skip to content
← All projects

DeployGuard

I wanted to see how far I could push cheap, interruptible infrastructure while keeping deployments safe.

  • Python
  • Kubernetes
  • k3s
  • Terraform
  • Prometheus
  • Trivy
  • AWS
Architecture diagram for DeployGuard
Tap or click to open full size

Demo

A real run on a local minikube cluster, with no cloud account, using only the four dg commands. Test traffic is sent by DeployGuard itself during the rollout. The last deploy ships a version that returns 500 errors, and DeployGuard stops it at 50% traffic and rolls back. The video plays at 1.5x, and long waits are sped up further where marked.

The problem

Deploying an app to Kubernetes from scratch means writing a Dockerfile, a stack of manifests, a CI pipeline and infrastructure. Each one is a place to get something subtly wrong: no resource limits, no readiness probes, a :latest tag, an over-permissive IAM role, or a NAT Gateway "added temporarily" that quietly costs about $32 a month.

Most people copy from an old repo or let an LLM generate the files and hope. I wanted a paved road instead, where the tool guards the generated configs, the cluster policy, the bill and the rollout. The name is literal.

Architecture

DeployGuard is a Typer CLI installed as dg. The workflow is four commands. dg init <name> renders a FastAPI service, its Dockerfile and its Kubernetes manifests from templates into a temporary folder, runs the guard on them, and only copies them into the project if the guard passes. dg cost parses the Terraform, prices it with Infracost and applies 8 cost rules. dg provision brings up a local minikube cluster, or a single k3s node on an AWS EC2 Spot instance through Terraform. dg deploy runs the guard again, builds an image tagged with the git commit, and hands it to the safety engine.

The safety engine has two stages. First, a temporary "green" copy of the deployment has to come up and answer a /readyz smoke test; it is removed afterwards, pass or fail. Then traffic moves to the new version through nginx-ingress canary weights, 10%, 50%, then 100%. At each step a Prometheus query checks the 5xx error rate, and if it crosses the threshold (1% by default) the canary is deleted and the stable deployment is rolled back.

What I built

  • dg, a Python CLI published on PyPI as dg-deploy and through a Homebrew tap, with the four-command workflow (init, cost, provision, deploy) plus doctor to check prerequisites and dashboard.
  • A two-stage safety engine: blue/green health pre-check, then canary rollout (10%/50%/100%) with automatic rollback on Prometheus error spikes.
  • A policy-as-code guard layer: kubeconform for schema validation, trivy config for misconfiguration scanning, and 7 custom rules (resource limits, probes, security context, no :latest tag, no root user, no privileged containers, IAM least privilege). Every finding says what is wrong, why it matters and how to fix it.
  • A cost gate that prices Terraform before provisioning and hard-stops over budget: dg cost exits non-zero when the estimate reaches the reject threshold or a reject rule fires, such as a NAT Gateway.
  • The Terraform for the cloud target: a VPC with one public subnet and no NAT Gateway, a t3.small Spot instance that installs k3s, an Elastic IP, and a pause path that scales the instance to zero.
  • A release pipeline: GitHub Actions publishes to PyPI on version tags through a Trusted Publisher (OIDC), with no stored PyPI token.

Decisions and trade-offs

k3s on EC2 Spot instead of a managed cluster. Managed Kubernetes was a non-goal for cost reasons. One Spot instance running k3s is enough for this workload, and I accepted the Spot interruption risk, which is also why demos run on the local target and AWS is the proof that it runs in the cloud too.

No NAT Gateway. The instance sits in a public subnet behind an Internet Gateway. A NAT Gateway's base fee alone is about $32 a month, more than the whole environment.

nginx-ingress instead of Traefik. Its canary-weight annotation was the simplest path to a 10/50/100 rollout, so k3s is installed with Traefik disabled.

Metrics fail open. If Prometheus is unreachable, the error-rate gate passes. Metrics being down should never abort a deploy; the /readyz smoke test is the hard gate.

The LLM drafts, the guard decides. dg init can draft files with a local model through LM Studio, but every draft goes through the same guard as the templates, and the templates are the fallback.

Libraries first. The guard, cost and deploy engines are importable modules, so the CLI and a Claude Code /guard skill share the same code.

What went wrong and how I fixed it

The package name was taken. deployguard already existed on PyPI. After a first rename and two failed publish runs, I settled on dg-deploy and updated every install reference.

Homebrew tried to compile Rust. The formula built pydantic-core from source, which pulled in a Rust toolchain. I first added the build dependency, then switched the formula to pre-built wheels to avoid the Rust build entirely.

The pre-check left its green deployment behind. After a successful pre-check, the temporary green copy was not cleaned up. I changed it so green is deleted whether the check passes or fails.

Ready replica counts stayed empty. readyReplicas on the deployment kept coming back null because of RBAC issues. I switched to counting pods through their owner references instead.

k3d kept timing out. Installing ingress-nginx with Helm on k3d timed out, so I moved the local target to minikube and its built-in ingress addon.

Results

  • dg-deploy 0.1.0 is published on PyPI, with a Homebrew tap.
  • 7 custom guard rules on top of kubeconform and Trivy.
  • 8 cost rules, with default thresholds of a warning at $10 a month and a hard stop at $50.
  • Canary steps of 10%, 50% and 100% by default, with rollback above a 1% 5xx error rate. Both are configurable.
  • The environment runs on k3s over AWS EC2 Spot for about $6/month.