Skip to main content

​
Secrets & Identity Management

This page describes the design of secret management and identity/access control across the Shokunin Platform. It covers how secrets are stored, who can access them, how they flow to the places that need them, and how new developers are onboarded.

​
Three Core Principles

Every decision in this design follows three rules:
  1. GCP Secret Manager is the single source of truth. No secret has an authoritative home anywhere else. GitHub Actions secrets, Vercel environment variables, and local .env files are all derived copies populated from Secret Manager.
  2. Terraform is the pipe, not the vault. Terraform reads values from Secret Manager (or from TF_VAR_* environment variables during bootstrapping) and distributes them outward. It never invents secret values.
  3. A human manually sets a secret exactly once. After initial creation, all distribution and rotation is automated.

​
Secret Namespaces

All secrets live in GCP Secret Manager in project shokunin-480309. They are organised into two namespaces by naming convention and access scope.

​
Platform namespace

Naming pattern: shokunin-{env}-{component} These secrets are shared across the team and consumed by CI/CD, Vercel runtime, and Cloud Run services.
Secret IDPurposeHow the value is set
shokunin-dev-dolt-db-passwordDolt MySQL root passwordTF_VAR_dolt_db_password on first terraform apply
shokunin-dev-beads-api-keyBeads API authentication keyTF_VAR_beads_api_key on first terraform apply
shokunin-dev-firebase-admin-keyFirebase Admin SDK service account JSONTerraform creates and stores the SA key automatically
shokunin-dev-firebase-browser-api-keyFirebase client SDK API keyHuman runs gcloud secrets versions add once
shokunin-dev-liveblocks-secret-keyLiveblocks server-side keyHuman runs gcloud secrets versions add once
shokunin-dev-eraser-api-tokenEraser diagram API token (shared)Human runs gcloud secrets versions add once
shokunin-dev-basehub-tokenBaseHub CMS token (shared)Human runs gcloud secrets versions add once
shokunin-dev-vercel-caller-sa-keySA key for Vercel → Cloud Run authTerraform creates and stores the SA key automatically
All platform secrets carry a standard label set: 
managed_by  = "terraform"
environment = "dev"
namespace   = "platform"
required    = "true" | "false"
consumer    = "vercel" | "cloud-run" | "local-dev" | "ci"
purpose     = "db-password" | "api-key" | "sa-key" | "sdk-token"
The required = "true" label drives monitoring alerts — any required secret with zero active versions triggers an alert.

​
Tenant namespace

Naming pattern: shokunin-{env}-{tenant_id}-{component} Tenant-scoped secrets are provisioned per tenant by the tenant Terraform module. They are isolated from platform secrets and from each other.

​
Identity Map

The table below describes every actor in the system, how it authenticates, and what it can access.
ActorAuthenticates asCan access
Human developerGCP user account (gcloud auth login)Secret Manager platform namespace (read-only); SA impersonation of dev-platform-sa for Terraform
Local Terraformdev-platform-sa via short-lived token (SA impersonation)Secret Manager (admin); GitHub via TF_VAR_github_token; Vercel via TF_VAR_vercel_token
GitHub Actions → GCPshokunin-dev-gha-sa via Workload Identity Federation (keyless)Artifact Registry (write); Cloud Run (deploy)
GitHub Actions → VercelVERCEL_TOKEN stored in GitHub secrets (populated by Terraform)Vercel API (deploy)
Vercel runtime → Cloud Runvercel-caller-sa JSON key in Vercel env var (populated by Terraform)Cloud Run service (invoke)
Cloud Run serviceshokunin-dev-run-sa workload identitySecret Manager (accessor on Dolt password, Beads API key); Dolt database
Dolt VMshokunin-dev-dolt-vm-sa workload identitySecret Manager (accessor on Dolt password)
Firebase Admin SDKshokunin-dev-firebase-admin SA key in Vercel env var (populated by Terraform)Firebase Authentication; Firestore admin
OpenCode agentopencode-agent SA (read-only)GCP project viewer, logging viewer, monitoring viewer

​
Service accounts

Every service account is managed in Terraform under platform/environments/foundation/ (project-level SAs) or platform/modules/iam/ (environment-level SAs).
Service accountPurpose
shokunin-dev-platform-saRuns Terraform for dev-shared and developer sandbox environments
shokunin-dev-tenant-saRuns Terraform for per-tenant resources
shokunin-dev-gha-saGitHub Actions CI/CD deployments (WIF, no key)
shokunin-dev-run-saCloud Run service identity
shokunin-dev-dolt-vm-saDolt VM identity
shokunin-dev-codespace-iapIAP tunnel access for developer containers
shokunin-dev-firebase-adminFirebase Admin SDK operations
shokunin-dev-vercel-caller-saVercel runtime → Cloud Run authentication
opencode-agentRead-only AI coding assistant diagnostics

​
Secret Flow Pipeline

GCP Secret Manager is the authoritative store. Terraform reads values from it and distributes them to every consumer. The local .env is populated by a developer script (scripts/env-sync) that reads from the same source. 
Human sets value once (gcloud secrets versions add  OR  TF_VAR_* env var)
        │
        ▼
GCP Secret Manager  ←── single source of truth
        │
        ├── [terraform apply] ──► GitHub Actions secrets
        │                          VERCEL_TOKEN
        │                          VERCEL_ORG_ID
        │                          VERCEL_PROJECT_ID
        │
        ├── [terraform apply] ──► Vercel environment variables
        │                          GOOGLE_SERVICE_ACCOUNT_KEY  (vercel-caller SA key)
        │                          GOOGLE_FIREBASE_ADMIN_KEY   (firebase-admin SA key)
        │                          NEXT_PUBLIC_FIREBASE_*      (browser SDK config)
        │                          LIVEBLOCKS_SECRET_KEY
        │                          BEADS_API_KEY
        │                          ERASER_API_TOKEN
        │                          BASEHUB_TOKEN
        │
        ├── [secret_key_ref]  ──► Cloud Run runtime env vars (at container start)
        │                          DOLT_PASSWORD
        │                          BEADS_API_KEY
        │
        └── [scripts/env-sync] ──► Local .env file
                                    Developer authenticates with their GCP user account.
                                    No manual secret handling required.

​
Bootstrap exceptions

Two secrets cannot be stored in Secret Manager because accessing Secret Manager itself requires a credential — a circular dependency. These are the only intentional exceptions to Principle 1:
SecretWhere it livesWhy
TF_VAR_github_tokenDeveloper’s shell environment onlyTerraform needs this to authenticate to GitHub to write GitHub secrets. Storing it in Secret Manager would require a working Terraform apply to retrieve it.
TF_VAR_vercel_tokenDeveloper’s shell environment onlySame reason — Terraform needs this to authenticate to Vercel before it can write Vercel env vars.
These are set by the operator before running terraform apply and are never committed or persisted: 
export TF_VAR_github_token="ghp_..."
export TF_VAR_vercel_token="..."
./terraform/scripts/tf dev-shared apply

​
Workload Identity Federation (WIF)

GitHub Actions authenticates to GCP without any stored credentials using Workload Identity Federation. The GitHub OIDC token is exchanged for a short-lived GCP access token at runtime. 
GitHub Actions workflow
        │  GitHub OIDC token
        ▼
GCP Security Token Service
        │  Short-lived access token
        ▼
shokunin-dev-gha-sa (impersonated)
        │
        ├── Artifact Registry (push Docker images)
        └── Cloud Run (deploy services)
WIF is configured in terraform/platform/environments/foundation/wif.tf. The pool is scoped to the Horizon-AI-dev GitHub organisation — tokens from other organisations are rejected.
WIF covers GitHub Actions → GCP only. Deployments from GitHub Actions to Vercel still require a VERCEL_TOKEN in GitHub secrets — Vercel does not currently support OIDC federation for inbound API authentication.

​
Terraform Modules

Secret and IAM management is split across four Terraform modules:
ModuleWhat it manages
platform/environments/foundation/State bucket; project-level SAs (dev-platform-sa, dev-tenant-sa, opencode-agent); WIF pool and provider; project IAM bindings
platform/modules/iam/Environment-level SAs (Cloud Run, Dolt VM, Codespace IAP, GHA, Firebase Admin, Vercel caller); IAM bindings scoped to resources
platform/modules/secret-manager/Secret Manager containers and versions for all platform secrets
platform/modules/github-secrets/Pushes secrets from Secret Manager → GitHub Actions secrets
platform/modules/vercel-env/Pushes secrets from Secret Manager → Vercel environment variables

​
SA key lifecycle

Two service account keys are managed entirely by Terraform — no manual gcloud iam service-accounts keys create steps:
  • vercel-caller-sa key — created in iam module, stored in shokunin-dev-vercel-caller-sa-key, pushed to Vercel as GOOGLE_SERVICE_ACCOUNT_KEY by the vercel-env module.
  • firebase-admin-sa key — created in iam module, stored in shokunin-dev-firebase-admin-key, pushed to Vercel as GOOGLE_FIREBASE_ADMIN_KEY by the vercel-env module.
To rotate a key: run terraform apply. Terraform replaces the google_service_account_key resource, updates Secret Manager, and re-pushes to Vercel in a single operation.

​
Developer Onboarding

Adding a new developer is a two-step process split between an admin and the developer.

​
Step 1 — Admin: grant bootstrap access

Create the developer’s identity file and grant them the permission needed to run their own sandbox terraform. 
# 1. Create the developer's tfvars (no secrets — identity only)
cat > terraform/config/developers/alice.tfvars <<EOF
developer_id    = "alice"
developer_email = "alice@horizon.ai"
EOF

git add terraform/config/developers/alice.tfvars
git commit -m "add developer: alice"
git push

# 2. Add alice@horizon.ai to developer_emails in foundation.tfvars, then apply
./scripts/tf foundation apply
Foundation grants serviceAccountTokenCreator on shokunin-dev-platform-sa to the developer’s Google account. This is the bootstrap permission that allows ./scripts/dev-setup to apply the developer’s own sandbox — without it, SA impersonation fails and dev-setup exits immediately.
serviceAccountTokenCreator lives in foundation (not in dev/main.tf) to break a circular dependency: dev/main.tf is applied via scripts/tf, which requires this very permission to impersonate the platform SA.

​
Step 2 — Developer: one-command local setup

# Authenticate with GCP (once per machine/container)
gcloud auth login
gcloud auth application-default login

# Run the setup script
./scripts/dev-setup alice alice@horizon.ai
dev-setup handles everything in order:
  1. Verifies prerequisites (gcloud, terraform, bun) and that gcloud is authenticated
  2. Creates config/developers/alice.tfvars if it does not already exist
  3. Runs ./scripts/tf dev alice apply — creates the developer’s Firestore database and grants Secret Manager read access
  4. Calls scripts/env-sync to populate .env from Secret Manager
  5. Runs a health check to verify the environment is working

​
scripts/env-sync

env-sync is the developer’s tool for keeping their local .env in sync with Secret Manager. It:
  • Authenticates using the developer’s own GCP user account (no SA key required)
  • Reads all secrets from the shokunin-dev-* platform namespace
  • Writes values to .env, skipping any that are already set
  • Prints a summary: what was updated, what was skipped, what is missing a version
# Re-sync at any time (safe to re-run)
./scripts/env-sync
env-sync requires gcloud auth login to have been run first. The developer’s GCP account must have been added to the platform by an admin (Step 1 above) before env-sync will have access to secrets.

​
Secret Monitoring

A Cloud Monitoring dashboard deployed via Terraform provides visibility into the state of all secrets:
  • All platform secrets are listed with their version count and last rotation date
  • Required secrets (required = "true" label) with zero active versions trigger an alert
  • Expiring secrets — any secret version with an expire_time set triggers an alert 30 days before expiry
The dashboard URL is output from Terraform after dev-shared apply. Developers can access it directly in the GCP Console under Monitoring → Dashboards.

​
Runbooks

​
Adding a new shared secret

  1. Create the Secret Manager container in platform/modules/secret-manager/main.tf
  2. Add the appropriate label (required, consumer, purpose)
  3. Add a vercel_project_environment_variable resource in platform/modules/vercel-env/main.tf if the secret is consumed by Vercel
  4. Run ./terraform/scripts/tf dev-shared apply
  5. Set the initial value: gcloud secrets versions add shokunin-dev-<name> --data-file=-

​
Rotating a secret

For secrets whose values are managed externally (not SA keys): 
# Add a new version (GCP automatically makes it the latest active version)
echo -n "new-value" | gcloud secrets versions add shokunin-dev-<name> --data-file=-

# Re-push to consumers
./terraform/scripts/tf dev-shared apply
For SA key secrets (vercel-caller-sa-key, firebase-admin-key): simply run terraform apply — Terraform recreates the key, updates Secret Manager, and re-pushes to Vercel.

​
Removing a developer’s access

Delete or remove the IAM binding in their developer .tfvars and re-apply. The developer’s GCP user account will lose secretmanager.secretAccessor on the platform namespace immediately.