This content originally appeared on DEV Community and was authored by 3deep5me
I recently created a free Kubernetes cluster on Oracles Always Free Tier. But how to handle secrets in a secure way, especially in a GitOps scenario?
This post will describe the full journey how to integrate the External Secrets Operator with 1Password to automatically inject secrets from 1Password to your cluster. For managing the different components, I will use argocd, but you can use also just helm install or FluxCD.
My Goal was to have a enterprise-known solution which integrates with different secret-managment tools, in my case it's 1Password.
External Secrets Operator
External Secrets Operator is a Kubernetes operator that integrates external secret management systems like AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, Azure Key Vault, IBM Cloud Secrets Manager, CyberArk Conjur and many more. The operator reads information from external APIs and automatically injects the values into a Kubernetes Secret.
-external-secrets.io
I like the fact that i can theoretically change my secret-management-tool without touching my secrets. Its also possible to have the same Secret-File in all my deployments over different stages. Because the secret-value gets filled regarding to the current stage. These are all things which were often a pain for me with sealed-secrets.
1Password
Really comfy Password-manager.
I have it so why not also using it for K8s?
Architecture
Let us have a look on the architecture especially the mechanics between external secret operator with Kubernetes and 1Password with the external secret operator.
External Secret Operator
The External Secret Operator uses in general two Custom Resources. The first is the SecretStore which holds information, how to connect to the "Secret Backend". The "Secret Backend" can be something like AWS Secrets Manager, Scaleway or like in our case 1Password. It holds for example the URL and the authentication for it. You can have different SecretStores if you have more than one provider. The other resource is the ExternalSecret which is a blueprint for the secret which should be created. So in practice if you create an ExternalSecret a normal K8s-Secret follows. The ExternalSecret holds information about which entry in 1Password to fetch and in which way to populate the values in a Secret. (More later)
1Password Connect
1Password-Connect is a kind-of Proxy which caches Secrets from the 1Password Cloud-Service, so you are able to retrieve secrets even 1Password is down, it also reduces the amount of requests to 1Password.
Its needed for the External Secrets Operator to work.
In my approach, it is right next to the External Secret Operator in the same namespace.
All together the workflow looks similar to that.
- External Secret is created
- External Secret Operator uses the information's in SecretStore to fetch the needed password/login from 1Password Connect
- 1Password connect itself fetch the needed password/login from 1Password directly
- External Secret Operator gets the password/login & creates the normal Kubernetes Secret with the requested values
Don't worry if you don't understand the whole process, we'll do a full hands-on walk-through at the end.
Install needed Helm-Charts
For the Installation of the helm-charts i will use argocd-apps, because i find this approach really portable. If i have a new cluster i just install argocd and then a kubectl apply . and i have all my needed apps installed. Beyond that i can use kustomize or helm and i have a auto-update feature of the apps - if i want to. Just set the targetRevision field to something like this 1.x.x.
But you can also use helm or fluxcd.
Install 1Password-Connect
Install the helm-chart
I modified the default values of the official Chart to the following:
# values.yaml
connect:
serviceType: ClusterIP
The default value is NodePort which is IMO dangerous if you publish your Secret-management-api to the local network or even the Internet. ClusterIP is suffient because the External Secrets Operator is in the cluster - not outside.
To install the Chart with the predefined values you can just apply this argocd-app. (kubectl apply -f app-1password-connect.yaml)
# app-1password-connect.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: 1password-connect
namespace: argocd
spec:
destination:
namespace: external-secrets
server: https://kubernetes.default.svc
project: default
source:
repoURL: https://1password.github.io/connect-helm-charts
targetRevision: 1.x.x
chart: connect
helm:
values: |
connect:
serviceType: ClusterIP
syncPolicy:
automated: {}
syncOptions:
- CreateNamespace=true
After that you should have a failed Pod in the external-secrets namespace.
$ kubectl get pods -n external-secrets
onepassword-connect-5fcbd4c68b-fgx68 0/2 CreateContainerConfigError 0 4d20h
The reason is Error: secret "op-credentials" not found (kubectl get event -n external-secrets). This Secret contains the token which is used for the connect-server to connect to 1Password.
To create the Secret install the 1Password CLI and execute following commands.
# Create a new Vault for the Secrets
$ op vault create "K8s"
# Create a connect server in 1Password
op connect server create "Kubernetes" --vaults "K8s"
The location of the credentials file can be seen in the output.
With the 1password-credentials.json in place we can create the needed secret.
# The connect service expects the 1password-credentials.json in base64
$ kubectl create secret generic op-credentials -n external-secrets --from-literal=1password-credentials.json="$(cat /path/to/1password-credentials.json | base64)"
After an while the pod should be running
$ kubectl get pod -n external-secrets
onepassword-connect-5fcbd4c68b-chmfg 2/2 Running 0
21m
Congratulations you completed the first step - installed the 1Password-Connect Server in Kubernetes!
Install the External Secret Operator
For installation you can use again an argocd-app.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: external-secrets-operator
namespace: argocd
spec:
destination:
namespace: external-secrets
server: https://kubernetes.default.svc
project: default
source:
repoURL: https://charts.external-secrets.io
targetRevision: 0.x.x
chart: external-secrets
syncPolicy:
automated: {}
syncOptions:
- CreateNamespace=true
After that your namespace should look like this.
$ kubectl get pods -n external-secrets
NAME READY STATUS RESTARTS AGE
external-secrets-operator-5db95c68b8-j6pzj 1/1 Running 1 (18d ago) 18d
external-secrets-operator-cert-controller-65f74dc7bf-784hg 1/1 Running 1 (18d ago) 18d
external-secrets-operator-webhook-5475bddd67-p9vq9 1/1 Running 0 18d
onepassword-connect-5fcbd4c68b-chmfg 2/2 Running 0 31m
Then we can establish the connection to the connect server (höhö).
# Create the token and save it in the OP_CONNECT_TOKEN environment variable
export OP_CONNECT_TOKEN=$(op connect token create "external-secret-operator" --server "Kubernetes" --vault "K8s")
# Create secret with the token which is used by the External-Secret-Operator ClusterSecretStore
kubectl create secret -n external-secrets generic onepassword-connect-token --from-literal=token=$OP_CONNECT_TOKEN
Create the ClusterSecretStore with kubectl apply -f
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
name: k8s
spec:
provider:
onepassword:
connectHost: http://onepassword-connect:8080
vaults:
K8s: 1 # look in this vault first
auth:
secretRef:
connectTokenSecretRef:
name: onepassword-connect-token
key: token
namespace: external-secrets
To see if it the connection was successful you can check the status of the ClusterSecretStore.
kubectl get -n external-secrets clustersecretstores.external-secrets.io k8s
NAME AGE STATUS CAPABILITIES READY
k8s 103m Valid ReadOnly True
Congratulations you also installed the external secret operator and established a connection with 1password connect.
End-to-End-Test the Solution
Theoretically you should be able to retrieve secrets from 1Password - but let us test it with a pragmatical example.
First let us create a secret in 1Password with the 1Password CLI.
# This creates a new entry named "Scaleway Credentails" with two properties/fields.
op item create --vault="K8s" --title="Scaleway Credentials" --category="login" accessKeyId="token-xyz" secretKey="xyz"
In the next step we can create our ExternalSecret which then creates our normal Kubernetes Secret.
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
# name of the ExternalSecret & Secret which gets created
name: scaleway-credentials
spec:
secretStoreRef:
kind: ClusterSecretStore
name: dev
target:
creationPolicy: Owner
data:
- secretKey: accessKeyId
remoteRef:
# 1password-entry-name
key: "Scaleway Credentials"
# 1password-field
property: accessKeyId
- secretKey: secretKey
remoteRef:
# 1password-entry-name
key: "Scaleway Credentials"
# 1password-field
property: secretKey
I tried my best to explain the references with comments.
To summarize the name of the ExternalSecret do not count. But its important that the secretKey is and the property field has the same value. The secretKey and property field represent the "Field" in 1Password e.g. password or user. The key field represents the entry in 1Password e.g. dev.to.
You can check the status of the external secret and also if the normal K8s secret was created.
$ kubectl get externalsecret -n default
NAME STORE REFRESH INTERVAL STATUS READY
scaleway-credentials k8s 1h SecretSynced True
$kubectl get secret -n default
NAME TYPE DATA AGE
scaleway-credentials Opaque 2 7m13s
You can see the secret is created. You can also have a deeper look into the values.
$ kubectl get secret -n default -o yaml
apiVersion: v1
items:
- apiVersion: v1
data:
accessKeyId: dG9rZW4teHl6
secretKey: eHl6
immutable: false
kind: Secret
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"external-secrets.io/v1beta1","kind":"ExternalSecret","metadata":{"annotations":{},"name":"scaleway-credentials","namespace":"default"},"spec":{"data":[{"remoteRef":{"key":"Scaleway Credentials","property":"accessKeyId"},"secretKey":"accessKeyId"},{"remoteRef":{"key":"Scaleway Credentials","property":"secretKey"},"secretKey":"secretKey"}],"secretStoreRef":{"kind":"ClusterSecretStore","name":"k8s"},"target":{"creationPolicy":"Owner"}}}
reconcile.external-secrets.io/data-hash: f3d79dd2ad3055723d0bebe3481f2372
creationTimestamp: "2024-07-29T15:36:04Z"
labels:
reconcile.external-secrets.io/created-by: c66895ad2c04350c73c512c0175cee24
name: scaleway-credentials
namespace: default
ownerReferences:
- apiVersion: external-secrets.io/v1beta1
blockOwnerDeletion: true
controller: true
kind: ExternalSecret
name: scaleway-credentials
uid: 1df5f0f5-ec44-4cb7-859a-3d3ef60b1d45
resourceVersion: "689603"
uid: 9543092a-7bdc-4695-9c83-bc35c36188c0
type: Opaque
kind: List
metadata:
resourceVersion: ""
Under data you can see the both keys and the corresponding values encoded in base64.
Congratulations you successfully created a secret in 1Password and pulled it into a normal Kubernetes Secret which can now be used as usual in deployments and other resources. 🥳🥳🥳
Thanks for reading my first post!
I hope you had fun and that it helped you. I'd love to hear your thoughts or feedback, so feel free to leave a comment below.
This content originally appeared on DEV Community and was authored by 3deep5me
3deep5me | Sciencx (2024-07-30T15:35:57+00:00) Using 1Password with External Secrets Operator in a GitOps way. Retrieved from https://www.scien.cx/2024/07/30/using-1password-with-external-secrets-operator-in-a-gitops-way/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.