Managing State, Secrets, and Environments in Terraform

The engineer who set up the Terraform project had left the company three months earlier. Nobody had thought to ask where the state file lived.

It lived on their laptop. Which IT had wiped.

Re-importing 23 GCP resources manually — Cloud Run services, Pub/Sub topics, Cloud SQL instances, IAM bindings — from a project with no consistent naming convention or resource tags took two days. Two days of terraform import, gcloud commands, and guessing which resource in the console matched which block in the config.

State, secrets, and environment configuration are the parts of Terraform that feel like infrastructure concerns and get deferred. They're the decisions that quietly determine whether your setup survives the team scaling, an engineer leaving, or a CI pipeline running on the wrong environment.

---

State

Terraform state is the source of truth for what exists. It maps your HCL config to real cloud resources. Without it, Terraform doesn't know what's deployed — it can't plan changes, detect drift, or destroy resources cleanly.

Local state (terraform.tfstate in your project directory) is a single point of failure. It can't be shared across a team. It disappears when a laptop dies or gets wiped. Never use it for anything beyond a personal experiment.

Remote state in GCS is the minimum viable setup:

# versions.tf
terraform {
  backend "gcs" {
    bucket = "pulsecart-terraform-state"
    prefix = "terraform/state"
  }
}

Create the bucket before running terraform init:

gsutil mb -l us-central1 gs://pulsecart-terraform-state
gsutil versioning set on gs://pulsecart-terraform-state

Versioning is what gives you state history. If a bad terraform apply corrupts state, you can restore a previous version from the bucket. Without versioning, that option doesn't exist.

GCS also handles state locking natively — when one engineer or CI job is running terraform apply, the state file is locked and other applies are blocked. No two applies can corrupt state simultaneously.

The three commands you'll need when something goes wrong:

# List all resources Terraform knows about
terraform state list

# Inspect a specific resource's state
terraform state show google_cloud_run_v2_service.producer

# Import an existing resource into state (when recovering from lost state)
terraform import google_cloud_run_v2_service.producer \
  projects/your-project/locations/us-central1/services/pulsecart-producer

terraform import is how you recover from the laptop situation — but it requires knowing the exact resource ID for every resource, which is why tagging and consistent naming conventions are worth enforcing from day one.

---

Secrets

Terraform state stores resource attributes in plaintext JSON. If you pass a database password as a variable and Terraform writes it to a resource, it's in your state file. In plaintext. In a GCS bucket that whoever has GCS access can read.

Two rules:

Never put secrets in tfvars files. prod.tfvars gets committed to Git. Even if it's in .gitignore today, it gets committed by accident eventually.

Read secrets from GCP Secret Manager at apply time:

# Read the DB password from Secret Manager during terraform apply
data "google_secret_manager_secret_version" "db_password" {
  secret  = "pulsecart-db-password"
  version = "latest"
}

resource "google_sql_database_instance" "pulsecart" {
  name             = "pulsecart-postgres"
  database_version = "POSTGRES_15"
  region           = var.gcp_region

  settings {
    tier = var.db_tier
  }
}

resource "google_sql_user" "app" {
  name     = "pulsecart_app"
  instance = google_sql_database_instance.pulsecart.name
  password = data.google_secret_manager_secret_version.db_password.secret_data
}

The secret value is fetched at apply time from Secret Manager — it never lives in your tfvars, your repo, or your CI environment variables. The only thing in Git is the secret name.

For variables that contain sensitive values, mark them explicitly:

# variables.tf
variable "db_password" {
  type      = string
  sensitive = true   # redacted from plan output and logs
}

sensitive = true prevents the value from appearing in terraform plan output or in CI logs. It doesn't prevent the value from being stored in state — which is why reading from Secret Manager rather than passing as a variable is the better pattern for anything genuinely sensitive.

---

Environments

Terraform Workspaces are the built-in answer to environments. They feel right — one codebase, multiple workspaces, each with isolated state. In practice they have a footgun: all workspaces share the same backend bucket prefix by default, just with a workspace name injected. It's easy to accidentally run terraform apply against the wrong workspace, especially in CI where the workspace is set by an environment variable.

Use tfvars files per environment instead:

environments/
├── dev.tfvars
├── staging.tfvars
└── prod.tfvars

# environments/dev.tfvars
gcp_project_id        = "pulsecart-dev"
environment           = "dev"
db_tier               = "db-g1-small"
db_availability_type  = "ZONAL"
min_instances         = 0
max_instances         = 5
deletion_protection   = false

# environments/prod.tfvars
gcp_project_id        = "pulsecart-prod"
environment           = "prod"
db_tier               = "db-custom-4-15360"
db_availability_type  = "REGIONAL"
min_instances         = 1
max_instances         = 20
deletion_protection   = true

Apply explicitly with the right var file:

# Dev
terraform apply -var-file="environments/dev.tfvars"

# Prod
terraform apply -var-file="environments/prod.tfvars"

In CI, the var file is determined by the branch or environment the pipeline runs against — explicit, auditable, hard to accidentally misapply.

Use workspaces when you have genuinely identical environments that differ only by region or account — multi-region deployments, for example. For dev/staging/prod, tfvars is clearer and safer.

---

Variable Validation

Catch misconfiguration at plan time, not after a broken apply:

# variables.tf
variable "environment" {
  type = string
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "environment must be one of: dev, staging, prod"
  }
}

variable "db_tier" {
  type = string
  validation {
    condition     = can(regex("^db-", var.db_tier))
    error_message = "db_tier must be a valid Cloud SQL tier starting with 'db-'"
  }
}

variable "min_instances" {
  type = number
  validation {
    condition     = var.min_instances >= 0 && var.min_instances <= var.max_instances
    error_message = "min_instances must be >= 0 and <= max_instances"
  }
}

terraform plan fails with a clear error message if validation fails. No apply, no partial deployment, no guessing what went wrong. One line of validation per constraint costs nothing and prevents a category of misconfiguration that's otherwise only caught in production.

---

The laptop incident was avoidable at every step — remote state from day one, a tagging policy, a runbook for what to do when someone leaves the team. None of it is complex. It's just the kind of setup that gets skipped when you're moving fast and the project is small.

It stops being small eventually.