k8s cert manager

cert-manager

它支持从Let’s Encrypt，HashiCorp Vault，Venafi等颁发证书

Issuer/ClusterIssuer

Certificate

ACME Orders and Challenges

Webhook

CA Injector

+-------------+
|   Ingress/  |
| annotations |
+------+------+
       |
       | watch ingress change
       |
       v
+-------------+
|   Issuer/   |
| ClusterIssuer |
+------+------+
       |
       | Create CertificateRequest
       |
       v
+------+------+
|CertificateRequest|
+------+------+
       |
       | Create Order
       |
       v
+------+------+
|      Order  |
+------+------+
       |
       | Create Challenges
       |
       v
+------+------+
|  Challenge  |
+------+------+
       |
       | Respond to Challenge
       |
       v
+------+------+
|ChallengeResponse|
+------+------+
       |
       | Issue Certificate
       |
       v
+------+------+
|     Secret  |
+------+------+

doc cert-manager

https://cert-manager.io/docs/installation/helm/

❯ helm repo add jetstack https://charts.jetstack.io
"jetstack" has been added to your repositories
❯ helm pull jetstack/cert-manager --untar
❯ kubectl apply -f ./cert-manager.crds.yaml   #--set installCRDs=true
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created

helm install -f ./cert-manager/values.yaml \
  -name cert-manager \
  --namespace cert-manager  \
  ./cert-manager  \
  --create-namespace  \
   --version v1.12.4 \
  # --set installCRDs=true
  --set prometheus.enabled=false \  # Example: disabling prometheus using a Helm parameter
  --set webhook.timeoutSeconds=4   # Example: changing the webhook timeout using a Helm parameter

helm install -f ./cert-manager/values.yaml -name cert-manager --namespace cert-manager  ./cert-manager  --create-namespace --set webhook.timeoutSeconds=4 --set installCRDs=true

❯ helm uninstall  -name cert-manager --namespace cert-manager
release "cert-manager" uninstalled

kpi

Vault Binary download for Linux

❯  helm repo add hashicorp https://helm.releases.hashicorp.com
"hashicorp" has been added to your repositories
❯ helm pull hashicorp/vault --untar

 helm install  -f ./vault/values.yaml  vault --namespace vault  ./vault  --create-namespace
 
 helm uninstall   vault --namespace vault

• failed to initialize barrier: failed to persist keyring: mkdir /vault/data/core: permission denied

  检查nfs配置权限 /etc/exports rw

  no_root_squash 选项会确保在 NFS 客户端上使用根用户（UID 为 0）

  容器中vault用户（属于other的权限），nfs宿主机目录权限设置 chmod go+rw -R ./vault/*

❯ kubectl exec vault-0 -n vault -- vault operator init \
    -key-shares=1 \
    -key-threshold=1 \
    -format=json >> ./vault/cluster-keys.json
❯ jq -r ".unseal_keys_b64[]" ./vault/cluster-keys.json
0pnsR3Z5j+lDXes6y6LaS0wh0IwuNn9FI/AMjsrR6w8=
❯ kubectl exec vault-0 -n vault -- vault operator unseal 0pnsR3Z5j+lDXes6y6LaS0wh0IwuNn9FI/AMjsrR6w8=

Key Value
-– -—-
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 1.14.0
Build Date 2023-06-19T11:40:23Z
Storage Type raft
Cluster Name vault-cluster-e7311051
Cluster ID b481f479-636b-44de-0133-0358e278547d
HA Enabled true
HA Cluster n/a
HA Mode standby
Active Node Address
Raft Committed Index 31
Raft Applied Index 31

vault operator init -key-shares=5 -key-threshold=3 
# -key-shares：指定密钥的总股数， 
# -key-threshold：指定需要几股可解锁 
# 以上参数为默认，可不设置

kubectl exec vault-0 -n vault -- vault status

https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-minikube-tls

kubectl exec -ti vault-1 -n vault -- vault operator raft join -address=http://vault-1.vault-internal:8200

https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/security-and-compliance/use-vault-as-a-key-management-service#3a460810ef6sd

vault全部都需要解封

❯ kubectl debug -it vault-2  -n vault  --image=k8s.org/cs/netshoot   -- sh
Defaulting debug container name to debugger-p6d5h.
If you don't see a command prompt, try pressing enter.
/root $ telnet vault-0.vault-internal:8201
telnet: can't connect to remote host (121.21.80.178): Connection refused
/root $ telnet vault-1.vault-internal:8201
Connected to vault-1.vault-internal:8201
/root $ telnet vault-2.vault-internal:8201
Connected to vault-2.vault-internal:8201

[ERROR] storage.raft: failed to heartbeat to: peer=vault-0.vault-internal:8201 backoff time=2.5s error=”dial tcp 121.21.80.178:8201: connect: connection refused”

2023-09-20T06:41:19.836Z [INFO] storage.raft: pipelining replication: peer=”{Voter vault-0 vault-0.vault-internal:8201}”

每次重启都要手动解封所有 vault 实例，自动解封autounseal-transit

实现 vault 配置的自动化管理

https://github.com/pulumi/pulumi-vault

path "local/*" {
  capabilities = ["read", "list"]
}

// 允许创建 child token
path "auth/token/create" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

traefik

https://www.boysec.cn/boy/393ed77e.html#%E6%96%B9%E6%B3%95%E4%BA%8C%EF%BC%88%E6%8E%A8%E8%8D%90%EF%BC%89

ingress

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    # The ACME server URL   pro https://acme-v02.api.letsencrypt.org/directory
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration 
    email: xxx@xx.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-staging
    # Enable the HTTP-01 challenge provider
    solvers:
      - http01:
          ingress:
            class:  nginx