安装 ArgoCD-谢先斌的博客

安装 ArgoCD 介绍

介绍

ArogCD 的组成：

argocd-server: ArgoCD gRPC/REST API server
argocd-repo-server: 是一个内部服务，它维护保存应用程序在Git中的本地缓存，并负责生成和返回Kubernetes资源清单
argocd-application-controller: 是一个Kubernetes控制器，它持续监视正在运行的应用程序，并将当前的活动状态与所需的目标状态进行比较，并保持为repo中定义的状态
argocd-dex-server：认证组件，由 https://github.com/dexidp/dex 实现

安装

高可用部署
指定版本安装，参考
在线示例

安装 argocd server

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

安装日志

argocd-install-event ...

$ kubectl create namespace argocd
$ kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
customresourcedefinition.apiextensions.k8s.io/applications.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/appprojects.argoproj.io created
serviceaccount/argocd-application-controller created
serviceaccount/argocd-applicationset-controller created
serviceaccount/argocd-dex-server created
serviceaccount/argocd-notifications-controller created
serviceaccount/argocd-redis created
serviceaccount/argocd-repo-server created
serviceaccount/argocd-server created
role.rbac.authorization.k8s.io/argocd-application-controller created
role.rbac.authorization.k8s.io/argocd-applicationset-controller created
role.rbac.authorization.k8s.io/argocd-dex-server created
role.rbac.authorization.k8s.io/argocd-notifications-controller created
role.rbac.authorization.k8s.io/argocd-server created
clusterrole.rbac.authorization.k8s.io/argocd-application-controller created
clusterrole.rbac.authorization.k8s.io/argocd-server created
rolebinding.rbac.authorization.k8s.io/argocd-application-controller created
rolebinding.rbac.authorization.k8s.io/argocd-applicationset-controller created
rolebinding.rbac.authorization.k8s.io/argocd-dex-server created
rolebinding.rbac.authorization.k8s.io/argocd-notifications-controller created
rolebinding.rbac.authorization.k8s.io/argocd-redis created
rolebinding.rbac.authorization.k8s.io/argocd-server created
clusterrolebinding.rbac.authorization.k8s.io/argocd-application-controller created
clusterrolebinding.rbac.authorization.k8s.io/argocd-server created
configmap/argocd-cm created
configmap/argocd-cmd-params-cm created
configmap/argocd-gpg-keys-cm created
configmap/argocd-notifications-cm created
configmap/argocd-rbac-cm created
configmap/argocd-ssh-known-hosts-cm created
configmap/argocd-tls-certs-cm created
secret/argocd-notifications-secret created
secret/argocd-secret created
service/argocd-applicationset-controller created
service/argocd-dex-server created
service/argocd-metrics created
service/argocd-notifications-controller-metrics created
service/argocd-redis created
service/argocd-repo-server created
service/argocd-server created
service/argocd-server-metrics created
deployment.apps/argocd-applicationset-controller created
deployment.apps/argocd-dex-server created
deployment.apps/argocd-notifications-controller created
deployment.apps/argocd-redis created
deployment.apps/argocd-repo-server created
deployment.apps/argocd-server created
statefulset.apps/argocd-application-controller created
networkpolicy.networking.k8s.io/argocd-application-controller-network-policy created
networkpolicy.networking.k8s.io/argocd-applicationset-controller-network-policy created
networkpolicy.networking.k8s.io/argocd-dex-server-network-policy created
networkpolicy.networking.k8s.io/argocd-notifications-controller-network-policy created
networkpolicy.networking.k8s.io/argocd-redis-network-policy created
networkpolicy.networking.k8s.io/argocd-repo-server-network-policy created
networkpolicy.networking.k8s.io/argocd-server-network-policy created

安装 argocd 客户端

brew install argocd

其他平台可以从 https://github.com/argoproj/argo-cd/releases 下载合适的可执行程序

export ARGOCD_OPTS='--port-forward-namespace argocd'
# argocd login <ARGOCD_SERVER>
argocd login localhost:8080

使用

服务暴露

Load Balancer

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

Port Forwarding

kubectl port-forward svc/argocd-server -n argocd 8080:443
# kubectl port-forward svc/argocd-server -n argocd 8080:80

通过 https://localhost:8080 访问

NodePort 方式

$ kubectl patch svc argocd-server -p '{"spec": {"type": "NodePort"}}'
$ kubectl get svc
NAME                                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)            AGE
...
argocd-server                             NodePort    10.245.94.10     <none>        80:31179/TCP,443:32007/TCP   19h
...

获取密码

默认用户admin

# 方式一
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

# 方式二
kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2

设置默认的 ns

kubectl config set-context --current --namespace=argocd

更新密码

argocd account update-password

cli 登录

$ argocd login localhost:8080
WARNING: server certificate had error: x509: certificate signed by unknown authority. Proceed insecurely (y/n)? y
Username: admin
Password:
'admin' logged in successfully
Context 'localhost:8080' updated

cli 认证

ARGOCD_SERVER - server address of argo cd
ARGOCD_AUTH_TOKEN - JWT auth token for requests
ARGOCD_OPTS - all other options: --grpc-web --insecure --plaintext

示例

示例一

name: guestbook
PROJECT: default
CLUSTER in-cluster (https://kubernetes.default.svc)
NAMESPACE default
REPO URL https://github.com/xiexianbin/argocd-example-apps.git
PATH kustomize-guestbook

kubectl port-forward svc/kustomize-guestbook-ui -n default 8081:80

示例二
创建示例项目

argocd app create guestbook --repo https://github.com/argoproj/argocd-example-apps.git --path guestbook --dest-server https://kubernetes.default.svc --dest-namespace default

Application 示例 application.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/argoproj/argocd-example-apps.git
    targetRevision: HEAD
    path: guestbook
  destination:
    server: https://kubernetes.default.svc
    namespace: guestbook

kubectl apply -n argocd -f application.yaml

项目和角色

project

kubectl create ns mynamespace
argocd proj create myproject -d https://kubernetes.default.svc,mynamespace -s https://github.com/argoproj/argocd-example-apps.git

说明：

k8s 的 namespaces 和 argocd 的 projects 和部署的仓库应该是分开的，限制权限
- project 使用 role 对项目的应用程序进行访问控制
- SOURCE REPOSITORIES：限制从哪里获取 k8s 资源清单
- DESTINATIONS：限制资源可以部署的位置，<k8s server>,<namespace_name>

# 获取 role
$ argocd proj role list default
ROLE-NAME        DESCRIPTION
devops           devops team
devops-readonly

# 获取role对应的 policy
$ argocd proj role get default devops
Role Name:     devops
Description:   devops team
Policies:
p, proj:default:devops, projects, get, default, allow
p, proj:default:devops, applications, *, default/*, allow
g, estack, proj:default:devops
p, proj:default:devops-readonly, projects, get, default, allow
p, proj:default:devops-readonly, applications, get, default/*, allow
g, estack, proj:default:devops-readonly
JWT Tokens:
ID  ISSUED-AT  EXPIRES-AT

# 生成 JWT
$ argocd proj role create-token default devops [-e 10m]
Create token succeeded for proj:default:devops.
  ID: a2f975e8-1438-4cfb-b9e3-972da24fffea
  Issued At: 202x-xx-xxT16:53:01+08:00
  Expires At: Never  # 默认永久有效，可以设置
  Token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.xxx.xxx

# 销毁
argocd proj role delete-token $PROJ $ROLE <id field from the last command>

使用

# 命令行方式
argocd --auth-token <JWT>

# 环境变量方式
ARGOCD_AUTH_TOKEN=<JWT>

debug

登录容器内部

kubectl exec -it argocd-application-controller-0 -- bash

调整日志级别

kubectl edit pod argo ..

  containers:
  - command:
    - /shared/argocd-dex
    - rundex
    - --log-level=debug

认证

argocd 通过 dex 对接 github 认证，其中 beehat 为 github 的 repo，还可以配置仓库的 team 实现权限的细化

apiVersion: v1
data:
  dex.config: |
    connectors:
      # GitHub example
      - type: github
        id: github
        name: GitHub
        config:
          clientID: 078b153a10d37bd090b4
          clientSecret: e2ea4f17c4254c5b9152f7fd373835b541c394dd
          orgs:
          - name: beehat
          loadAllGroups: false
          teamNameField: slug
          useLoginAsID: false
  url: https://137.184.7.93:31179

支持一个 organization 的所有仓库使用同一个认证信息：https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#repository-credentials

metadata.labels:
  - argocd.argoproj.io/secret-type: repository
  - argocd.argoproj.io/secret-type: repo-creds

权限 https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/#anonymous-access

$ kubectl edit secret argocd-secret -n argocd
...
stringData:
  # github webhook secret
  webhook.github.secret: shhhh! it's a GitHub secret

  # gitlab webhook secret
  webhook.gitlab.secret: shhhh! it's a GitLab secret

{"issuer":"https://kubernetes.default.svc.cluster.local","subject":"system:serviceaccount:argo:argo-server"}

https://argo-cd.readthedocs.io/en/stable/user-guide/projects/#configuring-rbac-with-projects

配置

kubectl edit configmap argocd-cm -n argocd

其他

Web-based Terminal
ApplicationSet 将应用部署到多个 k8s 环境中
认证模板为 https://github.com/argoproj 的认证会自动运用在 https://github.com/argoproj/xxx 的所有项目

argocd repocreds add https://github.com/argoproj --username youruser --password yourpass
argocd repocreds list

同步策略 Sync Policy

argocd app set <APPNAME> --sync-policy automated

spec:
  syncPolicy:
    automated: {}

配置自动同步策略时，不能使用 Rollback 功能

自动清理策略

argocd app set <APPNAME> --auto-prune

spec:
  syncPolicy:
    automated:
      prune: true

状态图标 Status Badge
为服务添加url

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-svc
  annotations:
    link.argocd.argoproj.io/external-link: http://my-grafana.com/pre-generated-link

通知可以添加 webhook
Notification subscriptions
参数覆盖，主要用在dev、test环境，不建议在生产环境使用

argocd app set guestbook -p image=example/guestbook:abcd123
argocd app sync guestbook

selective sync 针对大量资源同步耗时时，可以选择同步部分out-of-sync资源 argocd app set guestbook --sync-option ApplyOutOfSyncOnly=true
发布时Git监听策略
Tracking Kubernetes resources by label
Resource Hooks
- Sync Status with Jobs/Workflows with Time to Live (ttl)
- Using A Hook To Send A Slack Message
Argocd pull 检测周期：fast tracked using a webhook, or polled every 3 minutes
- Argo CD 每 3mins polls 部署仓库的变化，可以通过为 Gitlab/github 添加 webhook https://argocd.example.com/api/webhook 主动通知 Argo CD，参考
ApplicationSet Controller(2.3)：自动生成Argo CD Application
资源同步的优先级，wave越小应用越早，不同wave有2s的间隔：

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "5"

F&Q

argocd server 访问 redis 失败

错误日志

time="2022-05-31T04:04:11Z" level=warning msg="Failed to resync revoked tokens. retrying again in 1 minute: dial tcp 10.246.67.61:6379: i/o timeout"
time="2022-05-31T04:05:11Z" level=warning msg="Failed to resync revoked tokens. retrying again in 1 minute: dial tcp 10.246.67.61:6379: i/o timeout"

问题分析

经排查，是由于 network policy 策略生效（如安装 vcluster 等）导致的，临时删除：

kubectl -n argocd get networkpolicy
kubectl -n argocd delete networkpolicy argocd-server-network-policy argocd-redis-network-policy ...

修改 admin 密码

生成密码：

# bcrypt(password)=$2a$10$rRyBsGSHK6.uc8fntPwVIuLVHgsAhAX7TcdrqW/RADU0uh7CaChLa
kubectl -n argocd patch secret argocd-secret \
  -p '{"stringData": {
    "admin.password": "$2a$10$rRyBsGSHK6.uc8fntPwVIuLVHgsAhAX7TcdrqW/RADU0uh7CaChLa",
    "admin.passwordMtime": "'$(date +%FT%T%Z)'"
  }}'

hook 不能引用 output 输出

Unsupported conditions: outputs are not usable since LifecycleHook executes during execution time and outputs are not produced until the step is completed 参考

安装 ArgoCD

介绍

安装