Helm is to Kubernetes as yum and apt are to Linux. It’s the most mature and often de facto tool for Kubernetes package management. It makes installation of k8s software as simple as:
helm install <PACKAGENAME>Anthos Config Management (ACM) is one of Google’s services that does continuous deployment using GitOps methodologies. If you’re familiar with other GitOps tools, I would say that it most closely compares to ArgoCD or FluxCD.
Because I started my career in infrastructure and operations, one of the qualities that I appreciate most about ACM is that it was originally designed for platform operators. ACM can be used to bootstrap clusters and ensure they have desired security safeguards, compliance requirements, business rules, and business logic in a known good state.
For reference, Google Config Sync was open sourced in 2022 and its source is available here.
https://github.com/GoogleContainerTools/kpt-config-sync
Config Sync has supported public helm charts for some time via kustomize. Here’s a sample kustomize file that calls a public helm chart.
# ./base/kustomization.yaml
helmCharts:
- name: cert-manager
repo: https://charts.jetstack.io
version: v1.5.3
releaseName: my-cert-manager
namespace: cert-managerUnfortunately as of Jan 2023, kustomize does not yet support OCI compliant chart registries which until now, made it challenging to deploy helm charts from OCI compliant repositories such as Google Artifact Repository, JFrog Artifactory, etc.
To better support this use case, the ACM engineering team has recently introduced direct helm support.
ACM Config Sync Helm Deployer
The new helm deployer can be referenced within a RootSync or RepoSync as shown here.
- Create a RootSync or RepoSync object. Note that helm values can be overridden beneath `spec.helm.values`.
cat <<EOF>> root-sync-example.yamlapiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
name: root-sync-example
namespace: config-management-system
spec:
sourceFormat: unstructured
sourceType: helm
helm:
repo: oci://${AR_REGION}.pkg.dev/${PROJECT_ID}/${AR_REPO_NAME}
chart: mysql
version: 9.3.1
releaseName: my-mysql
namespace: test
auth: gcpserviceaccount
gcpServiceAccountEmail: ${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com
# From Anthos Config Management version 1.13.1 and later, you can use
# the optional field spec.helm.values to override default values.
# You can use the same format as the default values file to override
# default values.
values:
image:
pullPolicy: Always
primary:
resources:
limits:
cpu: 250m
memory: 256Mi
requests:
cpu: 250m
memory: 256Mi
EOF- Apply the RootSync object
kubectl apply -f root-sync.yaml- Verify the status of the root repository
nomos status --contexts=$(kubectl config current-context)
Connecting to clusters...
*gke_stolos-dev_us-central1-xxxxxx
--------------------
<root>:root-sync https://kubernetes.github.io/ingress-nginx/ingress-nginx:4.0.5
SYNCED ingress-nginx:4.0.5
Managed resources:
NAMESPACE NAME STATUS SOURCEHASH
clusterrole.rbac.authorization.k8s.io/my-ingress-nginx Current ingress
clusterrole.rbac.authorization.k8s.io/my-ingress-nginx-admission Current ingress
clusterrolebinding.rbac.authorization.k8s.io/my-ingress-nginx Current ingress
clusterrolebinding.rbac.authorization.k8s.io/my-ingress-nginx-admission Current ingress
ingressclass.networking.k8s.io/nginx Current ingress
namespace/ingress-nginx Current ingress
validatingwebhookconfiguration.admissionregistration.k8s.io/my-ingress-nginx-admission Current ingress
ingress-nginx configmap/my-ingress-nginx-controller Current ingress
ingress-nginx deployment.apps/my-ingress-nginx-controller Current ingress
ingress-nginx job.batch/my-ingress-nginx-admission-create Current ingress
ingress-nginx job.batch/my-ingress-nginx-admission-patch Current ingress
ingress-nginx role.rbac.authorization.k8s.io/my-ingress-nginx Current ingress
ingress-nginx role.rbac.authorization.k8s.io/my-ingress-nginx-admission Current ingress
ingress-nginx rolebinding.rbac.authorization.k8s.io/my-ingress-nginx Current ingress
ingress-nginx rolebinding.rbac.authorization.k8s.io/my-ingress-nginx-admission Current ingress
ingress-nginx service/my-ingress-nginx-controller Current ingress
ingress-nginx service/my-ingress-nginx-controller-admission Current ingress
ingress-nginx serviceaccount/my-ingress-nginx Current ingress
ingress-nginx serviceaccount/my-ingress-nginx-admission Current ingress
Private Repository
For a private repository, we may need to pass a token so that ConfigSync can access the repository. The additional steps to pass these credentials are detailed below.
- Create a secret with helm repo username and password
kubectl create secret generic foo \
--namespace=config-management-system \
--from-literal=username=helm-user \
--from-literal=password=helm-password- Create the RootSync and note the last few lines where we pass the auth/token information with the secret name.
apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
name: root-sync
namespace: config-management-system
spec:
sourceFormat: unstructured
sourceType: helm
helm:
#repo is the helm repository URL
repo: ${HELM_REPO_URL}
#chart is the helm chart name
chart: ${HELM_CHART_NAME}
version: ${HELM_CHART_VERSION}
releaseName: ${HELM_RELEASE_NAME}
namespace: ${HELM_RELEASE_NAMESPACE}
values:
resources:
requests:
cpu: 200m
memory: 250Mi
extraEnvs:
- name: NAME_1
value: "val1"
- name: NAME_2
value: "val2"
auth: token
secretRef: #<-- These lines
name: foo #<-- here- Apply the RootSync object
kubectl apply -f root-sync.yaml- Verify the status of the root repository
nomos status --contexts=$(kubectl config current-context)Google Workload Identity (with Google Cloud Source Repos)
The last scenario that we’ll show is synchronizing with Google Artifact Registry using a GCP Service Account.
- Ensure workload identity is enabled
- Assign the ‘artifactregistry.reader’ role to your Google Service Account
export PROJECT_ID=YOUR_PROJECT_ID
export GSA_NAME=YOUR_GSA_EMAIL_NAME
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member=serviceAccount:${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com --role=roles/artifactregistry.reader- Then the ‘iam.workloadIdentityUser’ role to your Google Service Account.
gcloud iam service-accounts add-iam-policy-binding --project=${PROJECT_ID} \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:${PROJECT_ID}.svc.id.goog[config-management-system/root-reconciler]" \
${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com- Configure the RootSync object and apply
apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
name: root-sync
namespace: config-management-system
spec:
sourceFormat: unstructured
sourceType: helm
helm:
#repo is the helm repository URL
repo: ${HELM_REPO_URL}
#chart is the helm chart name
chart: ${HELM_CHART_NAME}
version: ${HELM_CHART_VERSION}
releaseName: ${HELM_RELEASE_NAME}
namespace: ${HELM_RELEASE_NAMESPACE}
auth: gcpserviceaccount #<-- Service account type
gcpServiceAccountEmail: ${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com #<-- Service Acct- Apply the RootSync Object
kubectl apply -f root-sync.yaml- Verify the sync status of the root repository
nomos status --contexts=$(kubectl config current-context)Additional References for Further Exploration
ConfigSync w/ Helm via Kustomize:
https://cloud.google.com/anthos-config-management/docs/tutorials/config-sync-helm
ConfigSync w/ New Helm Deployer