Install Genesis with ArgoCD

Two charts, one sequence

Genesis is two Helm charts. genesis-ops installs first and genesis-platform second — in every model, without exception.

The alternatives further down deliver the same two charts a different way — as signed offline archives applied with the Genesis CLI, or with helm install directly. The charts, the ordering, and the values are identical either way.

ArgoCD / GitOps hybrid ★ Recommended

Deliver both the ops chart (CRDs + deploy-operator + genesis-bastion console + agents) and the platform chart through your own ArgoCD, from your own registry, driven by a values file you maintain in your own git repo — one file per environment. Four small files per environment drive the whole thing, and they're reproduced in full in 3.3a for you to copy.

Both charts install declaratively from your registry: genesis-ops first, then genesis-platform. The deploy-operator ships in the ops chart and short-circuits on installMethod: argocd (records a Skipped condition), so ArgoCD owns both installs end to end. Upgrades are the same motion as the first install: re-sync the new version into your registry (3.1), bump targetRevision in both Applications, commit, sync.

3.0 Prerequisites

• ArgoCD ≥ 2.8 (multi-source Application support — the mechanism this whole flow relies on: one source for the chart, a second for your values file). 2.8 is a hard floor, not a recommendation: preflight's argocd-version check FAILs below it. 2.6 and 2.7 do support multi-source but have known UI-diffing rough edges around it, which is why the floor sits where it does. 3.x is fine.
• RBAC on the argocd namespace. The identity that applies the Application needs create/get on applications.argoproj.io. This is the most common blocker — if your kubeconfig is denied on argocd, hand the generated manifests to an ArgoCD admin to apply.
• Your own registry (ACR / JFrog Artifactory / ECR / Harbor) with helm/OCI support, and your own git repo to hold the values files.
• A workload registry pull secret in both the ops and platform namespaces (managed by your ESO/Vault) that covers your registry.
• helm, skopeo and kubectl on whichever host you run the relocation and the applies from, plus the argocd CLI if you want to drive syncs from the command line.
• A short-lived pull credential for autonomizehub.azurecr.io, issued by your Autonomize contact — used once per release, for 3.1 only.

3.1 Relocate the release into your registry

Per release, copy both charts and every image they reference from Autonomize's distribution registry (autonomizehub.azurecr.io) into your own. This is a straight registry-to-registry copy — nothing is downloaded as a file, and your cluster never talks to autonomizehub.

Autonomize issues you a short-lived pull credential scoped to that registry — request one from your Autonomize contact. It's a plain username/password pair; nothing Azure-AD-specific is required on your side.

The charts:

VER=<version>                                # from `genesis releases`

helm registry login autonomizehub.azurecr.io \
  --username <autonomizehub-token-name> --password-stdin <<< "<autonomizehub-token-password>"
helm pull oci://autonomizehub.azurecr.io/helm/genesis-ops --version $VER
helm pull oci://autonomizehub.azurecr.io/helm/genesis     --version $VER

helm registry login <your-registry> \
  --username <your-registry-user> --password-stdin <<< "<your-registry-password>"
helm push genesis-ops-$VER.tgz oci://<your-registry>/helm
helm push genesis-$VER.tgz     oci://<your-registry>/helm

helm push always lands a chart at oci://<registry>/helm/<chart-name> — appending the chart's own name from its Chart.yaml. That's exactly the full chart-path the Applications expect (see the 401 note under 3.3), so there's nothing to reconcile by hand.

The images. Every image: reference the charts render is a copy target. Get the list from helm template against your own values, then loop:

# Enumerate what this release actually pulls, for both charts.
helm template oci://autonomizehub.azurecr.io/helm/genesis --version $VER \
  -f envs/prod/values.yaml \
  | grep -oE 'image: *"?[^"]+' | awk '{print $2}' | tr -d '"' | sort -u > images.txt

# Copy each one. --all is REQUIRED: without it skopeo tries to select a
# single platform variant matching the machine you run it from, which fails
# outright when copying Linux-only images from a Mac.
while read -r img; do
  skopeo copy --all \
    --src-creds "<autonomizehub-token-name>:<autonomizehub-token-password>" \
    --dest-creds "<your-registry-user>:<your-registry-password>" \
    "docker://$img" \
    "docker://<your-registry>/${img#*/}"
done < images.txt

There is no single "copy everything" command — script the loop from that image list rather than hand-copying, and re-run it per release.

3.2 Lay out your GitOps repo

Your repo needs four files per environment. Copy them from 3.3a below, or take the annotated versions from docs/customer/templates/ (argocd-application-ops.yaml, argocd-application-platform.yaml, argocd-ops-values.yaml, argocd-platform-values.yaml), and fill in your own values:

<your-gitops-repo>/
  envs/prod/genesis-ops-application.yaml       # ArgoCD Application — ops chart
  envs/prod/genesis-platform-application.yaml  # ArgoCD Application — platform chart
  envs/prod/ops-values.yaml                    # registry + pull secret + crds.install
  envs/prod/values.yaml                        # full platform values

Commit them to the repo ArgoCD will read. For each additional environment, copy envs/prod/ to envs/<env>/ and change the values path in both Application manifests plus the pinned chart version — one cluster and one ArgoCD per environment.

envs/<env>/values.yaml is the file you own and hand-edit going forward; everything else changes only on a version bump.

3.2a Values overrides this chart line needs

Beyond the registry, secrets and domain values above, a live ArgoCD install (unlike a plain helm install) surfaces a few gaps that are real values you need to set, not bugs to work around. Add these to envs/<env>/values.yaml as needed:

• genesis-de.temporal.server.config.namespaces.create: false — the chart's default namespace-bootstrap path has a hook whose PreSync Job needs a Sync-phase Service that ArgoCD hasn't created yet (see the FAQ below for why this is invisible under helm install). Setting false skips in-chart namespace creation; create the Temporal namespace once yourself after the first sync (tctl namespace register default, or your equivalent), same as any other one-time bootstrap step.
• global.secrets.externalSecrets.enabled — leave this true (the default) even when global.secrets.provider: kubernetes. It's the actual switch that suppresses the chart's internal secret-minting template (01-internal-secrets.yaml) — provider itself is ignored by that template's render gate, which only checks externalSecrets.enabled + global.secrets.azure.vaultUrl. Turning externalSecrets off to dodge an unrelated problem (below) instead leaves internal-secret-minting on, which under ArgoCD is worse: helm template's lookup is always empty, so every sync re-mints fresh random values for keys you already own — a silent, ongoing fight for ownership of the same Secret, not a one-time error.
• Every key global.secrets.data lists must exist in your vault, even for disabled optional components. The chart's ExternalSecret syncs the whole set as one all-or-nothing operation; one missing key (e.g. a client secret for an add-on you never enabled) fails the entire sync, which in turn stalls every later-wave Deployment behind it (same wave-gating mechanism as the Ingress FAQ entry). Fix at the source — provision a placeholder value for the unused key in your vault — rather than disabling externalSecrets (see above for why that trade is worse).
  Secrets management is the authoritative inventory: every key, which are required versus optional, the vault-key → Kubernetes-key mapping, and worked SecretStore configuration for Azure Key Vault, AWS Secrets Manager and GCP Secret Manager. Check it against your vault before the first platform sync — this is the cheapest place to catch a missing key.
• Bringing your own pre-existing ai-studio-secrets? The chart has no concept of "someone else already manages this Secret" — its own ExternalSecret and any Secret-owning mechanism you bring both target the exact same object name, and Kubernetes/ESO ownership is exclusive. If you provisioned the Secret yourself before adopting this chart, either let the chart's own ExternalSecret take over (delete your hand-made one; deletionPolicy: Retain means the underlying Secret's data survives) or keep yours and disable the chart's via externalSecrets.enabled: false and generateInternal: false — not just the former, or nothing creates the Secret at all and the chart refuses to render (a validation guard catches this combination specifically).
• genesis-gateway.gateway.routes.<component> — if you enable a component whose gateway route needs something the chart's built-in route entry for it doesn't have (extra HTTP methods, a header rewrite), set it here. The umbrella chart ships one static route table per component in genesis-gateway's own values, independent of that component's own routes block — see the FAQ if you hit a component whose declared route contract doesn't match what's actually served.
• <frontend-component>.env.KEYCLOAK_ISSUER / NEXT_PUBLIC_AUTH_KEYCLOAK_ISSUER — set these explicitly (to https://<your-domain>/auth/realms/<realm>) if your frontend component ends up with a duplicate-env-key manifest error on sync. See the FAQ for why.

3.3 Register credentials in ArgoCD (required — do this first)

Two credentials, for the two sources every exported Application has.

Source 1 — your registry. ArgoCD needs its own credential to pull either chart. Without it, sync fails with 401 unauthorized.

kubectl create secret generic genesis-helm-oci-creds -n argocd \
  --from-literal=type=helm \
  --from-literal=url=oci://<your-registry>/helm \
  --from-literal=enableOCI=true \
  --from-literal=username=<user> \
  --from-literal=password=<token> \
  --dry-run=client -o yaml \
  | kubectl label --local -f - argocd.argoproj.io/secret-type=repo-creds -o yaml \
  | kubectl apply -f -

Do not also create a secret-type=repository entry for the same registry — the credential template above matches both oci://<your-registry>/helm/genesis-ops and …/helm/genesis.

Source 2 — your GitOps repo. If it's private, register a standard ArgoCD git repository credential the normal way (argocd repo add <repo-url> --username ... --password ..., or an SSH key) — nothing Genesis-specific here.

3.3a Reference manifests

The four files your GitOps repo needs. Copy them as-is and replace <your-registry>, <your-gitops-repo-url>, <OPS_VERSION> and <PLATFORM_VERSION> throughout.

Each carries inline comments explaining what to replace and why the non-obvious settings are set the way they are.

envs/prod/genesis-ops-application.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: genesis-ops
  namespace: argocd
  labels:
    app.kubernetes.io/managed-by: genesis-cli
    genesis.autonomize.ai/install-method: argocd
    genesis.autonomize.ai/environment: prod
spec:
  project: default
  sources:
  # Source 1 — the ops chart, from YOUR registry. repoURL is the FULL chart
  # path (…/helm/genesis-ops), not the bare …/helm — see 3.3's 401 note.
  - repoURL: oci://<your-registry>/helm/genesis-ops
    chart: genesis-ops
    targetRevision: <OPS_VERSION>
    helm:
      releaseName: genesis-ops
      valueFiles:
      - $values/envs/prod/ops-values.yaml
  # Source 2 — YOUR GitOps repo, referenced only for its values file.
  # `ref: values` is what makes `$values/...` above resolve. No chart, no
  # helm block here.
  - repoURL: <your-gitops-repo-url>
    targetRevision: main
    ref: values
  destination:
    server: https://kubernetes.default.svc
    namespace: genesis
  syncPolicy:
    automated:
      prune: false          # values churn must never delete live resources
      selfHeal: true
    syncOptions:
    - CreateNamespace=true
    - ServerSideApply=true

envs/prod/ops-values.yaml — deliberately minimal. The ops chart needs only a registry, a pull secret, and the CRD toggle; everything else is chart default.

global:
  # Points every ops workload — Bastion, deploy-operator, agents — at the
  # images you synced in 3.1. This is the ArgoCD-path equivalent of the
  # wizard's `--set-variables IMAGE_REGISTRY=`.
  imageRegistry: <your-registry>
  imagePullSecrets:
    # Name only — the Secret itself is created by YOUR ESO/Vault in the
    # `genesis` namespace, never by this chart (we never create Secrets
    # holding customer credentials).
    - name: <your-pull-secret>

crds:
  # Leave true unless a cluster-admin pre-applied the 5 CRDs out-of-band.
  # If they did, set false so this Application doesn't need cluster-scoped
  # CRD create.
  install: true

envs/prod/genesis-platform-application.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: genesis-platform
  namespace: argocd
  labels:
    app.kubernetes.io/managed-by: genesis-cli
    genesis.autonomize.ai/install-method: argocd
    genesis.autonomize.ai/environment: prod
spec:
  project: default
  sources:
  - repoURL: oci://<your-registry>/helm/genesis
    chart: genesis
    targetRevision: <PLATFORM_VERSION>
    helm:
      releaseName: genesis-platform
      valueFiles:
      - $values/envs/prod/values.yaml
  - repoURL: <your-gitops-repo-url>
    targetRevision: main
    ref: values
  destination:
    server: https://kubernetes.default.svc
    namespace: genesis-platform
  syncPolicy:
    automated:
      prune: false
      selfHeal: true
    syncOptions:
    - CreateNamespace=true
    - ServerSideApply=true

envs/prod/values.yaml — the platform values file. This is the one file you own and hand-edit going forward. Abridged below to the blocks that matter for an ArgoCD install — see Configure for the full field reference.

global:
  imageRegistry: <your-registry>
  imagePullSecrets:
    - name: <your-pull-secret>

  domain: genesis.example.com          # matches your TLS cert
  ingress:
    className: nginx
    tlsSecret: genesis-tls

  secrets:
    # azure-keyvault | aws-secretsmanager | gcp-secretmanager | kubernetes
    provider: azure-keyvault
    # Leave TRUE even when provider is `kubernetes` — see 3.2a. This is the
    # switch that suppresses the chart's internal secret-minting template;
    # turning it off under ArgoCD causes re-minted random values every sync.
    externalSecrets:
      enabled: true
    azure:
      vaultUrl: https://<your-vault>.vault.azure.net
    # Every key listed here must EXIST in your vault — the ExternalSecret
    # syncs all of them as one all-or-nothing operation, and one missing key
    # stalls every later-wave Deployment behind it (3.2a).
    data:
      - DB_PASSWORD
      - REDIS_PASSWORD
      - KEYCLOAK_CLIENT_SECRET
      # … see the required-secrets reference for the full ~40-key list

database:
  host: <your-postgres>.postgres.database.azure.com
  port: 5432
  sslMode: require
  # Reference only — never a literal password.
  passwordRef:
    name: ai-studio-secrets
    key: DB_PASSWORD

redis:
  url: rediss://<your-redis>.redis.cache.windows.net:6380
  passwordRef:
    name: ai-studio-secrets
    key: REDIS_PASSWORD

# ── ArgoCD-specific overrides (3.2a) ──────────────────────────────────
genesis-de:
  temporal:
    server:
      config:
        namespaces:
          # The chart's namespace-bootstrap hook is a PreSync Job that needs
          # a Sync-phase Service ArgoCD hasn't created yet — a hard deadlock
          # under ArgoCD's hook-phase model, invisible under `helm install`.
          # Register the Temporal namespace once by hand after first sync.
          create: false

3.4 Apply the genesis-ops Application

kubectl apply -f envs/prod/genesis-ops-application.yaml
argocd app sync genesis-ops           # force the first reconcile now
argocd app wait genesis-ops --health

Wait for it to be healthy before continuing:

# all 5 CRDs served
kubectl get crd | grep genesis.autonomize.ai

# operator + console Ready in the ops namespace
kubectl -n genesis get deploy deploy-operator genesis-bastion

Only once both are Ready does the platform Application have the CRDs and operator it depends on.

3.5 Run preflight (optional)

With the ops chart installed, the preflight runner is available in the cluster. Running it here catches a missing prerequisite — unreachable Postgres, absent Redis, an ESO SecretStore that doesn't resolve, a missing TLS cert — before the platform sync starts creating workloads that will crash-loop on it instead.

First, something has to write the configuration preflight reads. The checks are driven by a genesis-platform-config ConfigMap, and the chart does not render it. On this path the Genesis Bastion console writes it — the CLI is not involved unless you choose it.

kubectl -n genesis port-forward svc/genesis-bastion 8020:80

Open the Configuration tab, fill it in and save — that writes the ConfigMap in-cluster. Then open the Preflight tab and run it. The console's install screens are being retired; its Configuration and Preflight steps are kept precisely so this works without a CLI.

Three other ways to trigger the same checks:

• Apply a PreflightReport CR and read its status. The deploy-operator watches the kind and runs the Job.
• Let every sync run it. The chart can render the CR and its Job as an ArgoCD PostSync hook — no console, no CLI, no operator. It is off by default; enable it in your ops values and read the report after any sync:

  preflightJob:
    postSync:
      enabled: true

  kubectl -n genesis get preflightreport postsync-preflight \
    -o jsonpath='{.status.phase}'

• The genesis CLI, if you already have it on a host with a kubeconfig:

  genesis configure --save    # writes the same ConfigMap
  genesis preflight

Read the table rather than the exit code: SKIP is a legitimate verdict (the bundled IdP isn't installed until 3.6, so its check can't run yet), and a WARN-only report still exits 0. No check should report FAIL.

Or skip the step entirely and rely on the prerequisites checklist.

3.6 Apply the genesis-platform Application

kubectl apply -f envs/prod/genesis-platform-application.yaml
argocd app sync genesis-platform      # force the first reconcile now
argocd app wait genesis-platform --health

The deploy-operator short-circuits on installMethod=argocd (records a "Skipped" condition) — ArgoCD owns the platform install end to end.

3.7 Day-2: values changes and release upgrades

• Config change: edit envs/<env>/values.yaml directly, commit, push. ArgoCD's selfHeal picks it up (or argocd app sync genesis-platform to force it now). syncPolicy.automated.prune stays false — values churn never deletes live resources.
• Release upgrade: repeat 3.1 for the new version, then bump targetRevision in both Application files to match. Commit, then argocd app sync genesis-ops genesis-platform. Nothing else changes — values.yaml carries forward untouched.

3.8 Troubleshooting & FAQ

Common sync errors

401 on sync. Almost always the full-chart-path rule in 3.3 — confirm your repo-creds Secret's url is oci://<your-registry>/helm (a prefix) and the Application's repoURL is the full chart path (…/helm/genesis-ops or …/helm/genesis), never the bare …/helm.

APISIX CRDs. Helm applies a chart's special crds/ directory only on first install — ArgoCD's sync has the same behavior. Syncing onto a namespace with a prior release skips them:

no matches for kind "ApisixRoute" in version "apisix.apache.org/v2" — ensure CRDs are installed first

Fix — apply the APISIX CRDs cluster-wide once, before the first sync (kubectl get crd | grep apisix to check; extract from the gateway subchart's crds/ if missing).

Registry credential expiry. A short-lived token (e.g. az acr login --expose-token, ~3h, or the 3.1a relocation token) breaks re-syncs once it lapses — see 3.3's "durable registry credential" note. The 3.1a token is meant to be thrown away after the one-time copy, not reused here.

Known chart-line issues that need an out-of-band (non-values) fix

Everything in 3.2a is a values override — normal config, no manual cluster surgery involved. The two issues below are different in kind: they're real chart-template defects, invisible under a plain helm install, that ArgoCD's stricter ordering exposes. Both are being fixed upstream; until that fix ships, here's the workaround.

Schema/migration Job stuck forever on "ConfigMap not found." Some migration Jobs run as an ArgoCD PreSync hook and mount a ConfigMap that the same chart renders as a plain (non-hook) resource. PreSync hooks run strictly before the whole Sync phase — so that ConfigMap genuinely does not exist yet when the Job starts. This is invisible under helm install (Helm creates everything in one pass, ConfigMaps before Jobs by its fixed kind-sort order, and Jobs just retry via their normal backoffLimit) but is a hard, permanent deadlock under ArgoCD's hook-phase model. Workaround: apply the affected ConfigMap directly before the first sync (or after any release bump that touches it):

kubectl create -f <(helm template oci://<your-registry>/helm/genesis --version <ver> \
  -f envs/<env>/values.yaml --show-only <path-to-the-configmap-template>) -n <namespace>

Re-check after each version bump — the fix is expected in a future chart release, at which point this step becomes a no-op you can drop.

A later-wave Deployment never gets created; ArgoCD sits at "waiting for healthy state of ... Ingress" indefinitely. Only relevant if you're not using the chart's built-in Ingress (e.g. you front the cluster with a Gateway API Gateway/HTTPRoute instead, because your ingress controller isn't the nginx-compatible one the chart assumes). The chart's own Ingress resource has no independent toggle to disable it — it renders whenever global.routing.mode is set — and if nothing actually reconciles it, it never gets a status.loadBalancer.ingress entry. ArgoCD's default health check for Ingress waits on exactly that field, and sync-wave progression blocks on every resource in a wave being Healthy before the next wave starts — so an Ingress stuck Progressing forever silently stalls everything scheduled after it, with no error, just an indefinite Running operation. Workaround: patch the Ingress's status directly with a placeholder — this is a status-only field with zero effect on real traffic (your actual routing is the separate Gateway/HTTPRoute object):

kubectl patch ingress <ingress-name> -n <namespace> --subresource=status --type=merge \
  -p '{"status":{"loadBalancer":{"ingress":[{"hostname":"routed-via-gateway.invalid"}]}}}'

Ongoing housekeeping (not bugs — just how prune: false behaves)

PreSync-hook resources pile up over time. Every hook Job/ConfigMap (schema migrations, etc.) stays in the namespace once it's run — ArgoCD doesn't clean these up automatically, and syncPolicy.automated.prune stays false by design (3.6: values churn should never delete live resources). They're harmless (spent, one-shot, not referenced by anything running) but do show up as permanent OutOfSync entries with sync phase None. Delete them whenever it's noisy enough to matter:

kubectl get application genesis-platform -n argocd -o json \
  | jq -r '.status.resources[] | select(.status=="None") | "\(.kind) \(.name)"'
# then kubectl delete <kind> <name> -n <namespace> for each

A resource can show OutOfSync in argocd app get/the UI with nothing actually different. Before spending time on it, confirm with the authoritative comparison rather than the cached status tree:

argocd app diff genesis-platform

An empty diff plus Health: Healthy means it's a display artifact for that specific resource, not real drift — seen with ExternalSecret objects whose owning controller (ESO) refreshes status independently of Helm/ArgoCD's render. ignoreDifferences on /status does not fix this particular case (already tried) — it's cosmetic, safe to ignore.

3.9 Post-install + when it breaks

genesis doctor -n genesis          # one verdict: what's wrong + the fix

Full troubleshooting flow + support-bundle handoff: see the CLI install guide.

3.10 Deprecated: the console's inline export

The Bastion console's Configuration tab still has an "Export ArgoCD Application" screen (single-source, values inlined as valuesObject, platform-only — no ops Application, no per-environment values file in your own repo). It still works and nothing here removes it, but it's no longer the recommended path: it bakes your config into the Application itself (harder to diff/review as a values change) and never covered the ops chart at all. New installs should use 3.0–3.6 above; existing installs on the inline export can migrate at their own pace — the underlying chart/values are unaffected either way, so there's no forced cutover.

Other deployment methods

For clusters where ArgoCD isn't an option. Each produces the same end state as the ArgoCD path above.

# On the gap-host (one-way internet to the portal)
genesis login                                   # paste your sk_... license key
VER=<version>                                # from `genesis releases`
genesis pull $VER -o /tmp/bundles                # both bundles + .sig/.sha256/SBOM
genesis verify /tmp/bundles/genesis-ops-$VER.tar.zst
genesis verify /tmp/bundles/genesis-platform-$VER.tar.zst
genesis push-images --bundle /tmp/bundles/genesis-ops-$VER.tar.zst      --to your-registry.azurecr.io
genesis push-images --bundle /tmp/bundles/genesis-platform-$VER.tar.zst --to your-registry.azurecr.io

# Cross the gap (USB / encrypted SCP), then on the cluster-side host:
genesis verify ./genesis-ops-$VER.tar.zst        # re-verify after transfer
# Deploy ops FROM YOUR REGISTRY — images come through your scanner:
zarf package deploy ./genesis-ops-$VER.tar.zst \
  --set-variables IMAGE_REGISTRY=your-registry.azurecr.io \
  --set-variables IMAGE_PULL_SECRET=your-pull-secret \
  --confirm
genesis configure --save --emit-helm-values /tmp/values.yaml
genesis preflight                                # no check may FAIL (SKIP is fine)
genesis deploy --bundle ./genesis-platform-$VER.tar.zst \
  --registry your-registry.azurecr.io --values /tmp/values.yaml

# Pre-apply the CRDs (requires only CRD create permission).
#
# Step 1 — get the ops CHART out of the zarf bundle. The bundle is a
# zstd-compressed tar; the chart is a .tgz nested inside a component tar.
#
# Requires the `zstd` binary (apt-get install zstd / dnf install zstd).
# GNU tar's --zstd flag shells out to the same binary, so it is not an
# alternative. Piping means the bundle's multi-GB image layers are never
# written to disk — only the few MB of chart. It still READS the whole
# stream, so expect this to take a few minutes on a large bundle.
#
# Member paths below are LITERAL on purpose. `components/*.tar` fails on
# GNU tar ("Pattern matching characters used in file names", exit 2)
# unless you add --wildcards, and the ops bundle has three components,
# so a shell glob in step 2 opens the wrong one.
mkdir -p /tmp/genesis-ops-chart && cd /tmp/genesis-ops-chart
zstd -dc /tmp/genesis-ops-$VER.tar.zst | tar -xf - components/ops-chart.tar
tar -xf components/ops-chart.tar
CHART=$(ls */charts/genesis-ops-*.tgz | head -1)
echo "chart: $CHART"

# (Alternative, no zstd binary needed: `zarf tools archiver decompress
#  /tmp/genesis-ops-$VER.tar.zst /tmp/genesis-ops` — but it unpacks the
#  ENTIRE bundle, images included, which needs many GB of free disk.)

# Step 2 — render the CRDs. No cluster access, no values, no credentials.
#
# --kube-version is REQUIRED: the chart pins kubeVersion >=1.28.0-0, and
# `helm template` checks that against your helm BINARY's built-in default,
# never against your cluster. Without it, helm <=3.12 fails with a
# confusing "incompatible with Kubernetes v1.26.0".
#
# `kubectl apply -f chart/templates/crds/` does NOT work: the CRDs are Helm
# templates carrying a `{{- if }}` guard, not plain manifests.

# Apply CRDs (requires only apiextensions.k8s.io/customresourcedefinitions: create)
helm template "$CHART" --show-only 'templates/crds/*' --kube-version 1.28.0 \
  | kubectl apply -f -

# Verify all 5 are served
kubectl get crd | grep genesis
#   genesisdeployments.genesis.autonomize.ai
#   genesisupgrades.genesis.autonomize.ai
#   preflightreports.genesis.autonomize.ai
#   healthreports.genesis.autonomize.ai
#   workflowsmokereports.genesis.autonomize.ai

# Step 2 — deploy ops bundle (no CRD create needed; they exist).
# Registry-first (scanner-gated) — same as the standard model:
zarf package deploy ./genesis-ops-$VER.tar.zst \
  --set-variables IMAGE_REGISTRY=your-registry.azurecr.io \
  --set-variables IMAGE_PULL_SECRET=your-pull-secret \
  --confirm
# Or bare Zarf (unscanned in-cluster registry) — pass the LOCAL init
# package you transferred; a bare `zarf init` pulls from ghcr.io and fails
# air-gapped:
#   zarf init ./zarf-init-amd64-<ZARF_VER>.tar.zst --confirm
#   zarf package deploy genesis-ops-$VER.tar.zst \
#     --set-variables IMAGE_REGISTRY=<your-registry> --confirm

# Decompress bundle
zstd -d genesis-platform-$VER.tar.zst -o genesis-platform-$VER.tar
tar -xf genesis-platform-$VER.tar -C /tmp/genesis-bundle/

# Mirror images into your registry FROM THE BUNDLE — they ship inside the
# .tar.zst. Never pull from the vendor registry: it is unreachable from an
# air-gapped cluster, which is the whole reason you are on this path.
genesis push-images --bundle genesis-platform-$VER.tar.zst \
  --to your-registry.internal
genesis push-images --bundle genesis-ops-$VER.tar.zst \
  --to your-registry.internal
# (equivalent manual loop: skopeo copy from the extracted bundle's OCI
#  image layout — oci:/tmp/genesis-bundle/images:<ref> — to your registry)

# Install ops workloads.
#
# deployOperator.zarfStateNamespace: "" is REQUIRED on this path, on the
# first install AND on every upgrade. The chart grants the operator read
# access to zarf's state Secret through a Role in the `zarf` namespace —
# correct when zarf installed the chart, but here you never run
# `zarf init`, so that namespace does not exist and a Role targeting it
# fails the WHOLE helm operation with:
#
#   Error: namespaces "zarf" not found
#
# Put it in your values file rather than passing --set: helm does not carry
# a --set forward on upgrade unless you also pass --reuse-values, so a flag
# has to be remembered every time and a values file does not.
#
#   # /tmp/genesis-platform-values.yaml
#   deployOperator:
#     zarfStateNamespace: ""
helm install genesis-ops /tmp/genesis-bundle/chart/ \
  --namespace genesis --create-namespace \
  --values /tmp/genesis-platform-values.yaml

# Install platform
helm install genesis-platform /tmp/genesis-bundle/chart/ \
  --namespace genesis \
  --values /tmp/genesis-platform-values.yaml

See also