Skip to content
Published on

ArgoCD ApplicationSet 与 Progressive Rollout：GitOps 多集群部署实战指南

🇰🇷KO🇺🇸EN🇯🇵JA🇨🇳ZH
필사 모드
分享
Authors

本文是本系列的多集群运维篇。如果你想先通览 Blue-Green 与 Canary 的整体决策框架、自动回滚以及基于 Istio 的流量控制，建议先读 用 ArgoCD + Argo Rollouts 实现的 GitOps 部署策略 — Blue-Green、Canary 完全指南。这里则围绕 ApplicationSet、Generator、Progressive Sync 与环境 fan-out 运维展开。

1. 什么是 ArgoCD ApplicationSet

ApplicationSet 是 ArgoCD 的扩展资源，它从一份模板 自动生成多个 Application。在多集群、多租户、单体仓库环境中，与其手工管理几十到几百个 Application，不如用它做到 声明式自动化。

本文想回答的问题，与其说是“Canary 该怎么做”，不如说是“当集群和环境变多之后，如何把 GitOps 部署维持在一个可运维的结构里”。也就是说，部署策略的大图交给中枢文章，这里专注于把该策略安全地铺开到多个环境的运维装置。

1.1 为什么要用 ApplicationSet？

传统方式:
  app-dev.yaml    → Application (dev)
  app-staging.yaml → Application (staging)
  app-prod.yaml   → Application (prod)
  ... × 50 个服务 = 150 个 YAML 手动管理 😱

ApplicationSet:
  appset.yaml → 用一份模板自动生成 150 个 ✅

2. ApplicationSet Generator 类型

2.1 List Generator

最简单的形态。从显式的参数列表生成 Application：

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapp-envs
  namespace: argocd
spec:
  generators:
    - list:
        elements:
          - cluster: dev
            url: https://dev-k8s.example.com
            namespace: myapp-dev
          - cluster: staging
            url: https://staging-k8s.example.com
            namespace: myapp-staging
          - cluster: prod
            url: https://prod-k8s.example.com
            namespace: myapp-prod
  template:
    metadata:
      name: 'myapp-{{cluster}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/org/myapp.git
        targetRevision: HEAD
        path: 'k8s/overlays/{{cluster}}'
      destination:
        server: '{{url}}'
        namespace: '{{namespace}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

2.2 Git Generator (Directory)

在单体仓库结构中基于目录自动发现：

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-addons
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: https://github.com/org/cluster-addons.git
        revision: HEAD
        directories:
          - path: addons/*
          - path: addons/exclude-this
            exclude: true
  template:
    metadata:
      name: '{{path.basename}}'
    spec:
      project: addons
      source:
        repoURL: https://github.com/org/cluster-addons.git
        targetRevision: HEAD
        path: '{{path}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{path.basename}}'

2.3 Cluster Generator

自动部署到 ArgoCD 中已注册的所有集群：

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: monitoring-stack
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            env: production
  template:
    metadata:
      name: 'monitoring-{{name}}'
    spec:
      project: infrastructure
      source:
        repoURL: https://github.com/org/monitoring.git
        path: helm-chart
        helm:
          valueFiles:
            - 'values-{{metadata.labels.region}}.yaml'
      destination:
        server: '{{server}}'
        namespace: monitoring

2.4 Matrix Generator（组合）

把两个 Generator 交叉组合：

spec:
  generators:
    - matrix:
        generators:
          - git:
              repoURL: https://github.com/org/apps.git
              revision: HEAD
              directories:
                - path: apps/*
          - clusters:
              selector:
                matchLabels:
                  env: production

3. Progressive Sync：分阶段推出

3.1 概念

Progressive Sync 是一种让 ApplicationSet 生成的那些 Application 不一次性全部部署、而是分阶段部署 的策略。可以按 QA → Staging → Production 的顺序边验证边推进。

3.2 配置

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapp-progressive
  namespace: argocd
spec:
  generators:
    - list:
        elements:
          - cluster: qa
            url: https://qa-k8s.example.com
          - cluster: staging
            url: https://staging-k8s.example.com
          - cluster: prod-east
            url: https://prod-east-k8s.example.com
          - cluster: prod-west
            url: https://prod-west-k8s.example.com
  strategy:
    type: RollingSync
    rollingSync:
      steps:
        - matchExpressions:
            - key: cluster
              operator: In
              values:
                - qa
          # 先部署 QA → 自动 Sync
        - matchExpressions:
            - key: cluster
              operator: In
              values:
                - staging
          maxUpdate: 100%
          # QA 成功后 staging 自动进行
        - matchExpressions:
            - key: cluster
              operator: In
              values:
                - prod-east
                - prod-west
          maxUpdate: 50%
          # prod 按每次 50% 分阶段进行
  template:
    metadata:
      name: 'myapp-{{cluster}}'
      labels:
        cluster: '{{cluster}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/org/myapp.git
        targetRevision: HEAD
        path: 'k8s/overlays/{{cluster}}'
      destination:
        server: '{{url}}'
        namespace: myapp
      syncPolicy:
        automated:
          prune: true

4. Argo Rollouts：Canary 与 Blue-Green

4.1 Canary 发布

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: myapp
spec:
  replicas: 10
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - setWeight: 30
        - pause: { duration: 5m }
        - analysis:
            templates:
              - templateName: success-rate
            args:
              - name: service-name
                value: myapp
        - setWeight: 60
        - pause: { duration: 10m }
        - setWeight: 100
      canaryService: myapp-canary
      stableService: myapp-stable
      trafficRouting:
        istio:
          virtualServices:
            - name: myapp-vsvc
              routes:
                - primary
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: myapp:v2
          ports:
            - containerPort: 8080

4.2 AnalysisTemplate（自动回滚）

apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  args:
    - name: service-name
  metrics:
    - name: success-rate
      interval: 30s
      count: 10
      successCondition: result[0] >= 0.95
      failureLimit: 3
      provider:
        prometheus:
          address: http://prometheus.monitoring:9090
          query: |
            sum(rate(http_requests_total{
              service="{{args.service-name}}",
              status=~"2.."
            }[5m])) /
            sum(rate(http_requests_total{
              service="{{args.service-name}}"
            }[5m]))

4.3 Blue-Green 发布

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: myapp-bluegreen
spec:
  replicas: 5
  strategy:
    blueGreen:
      activeService: myapp-active
      previewService: myapp-preview
      autoPromotionEnabled: false
      prePromotionAnalysis:
        templates:
          - templateName: success-rate
      scaleDownDelaySeconds: 300
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: myapp:v2

5. 实战：ApplicationSet + Rollouts 整合

5.1 目录结构

repo/
├── apps/
│   ├── frontend/
│   │   ├── base/
│   │   │   ├── rollout.yaml
│   │   │   ├── service.yaml
│   │   │   └── kustomization.yaml
│   │   └── overlays/
│   │       ├── dev/
│   │       ├── staging/
│   │       └── prod/
│   └── backend/
│       ├── base/
│       └── overlays/
└── appsets/
    └── all-apps.yaml

5.2 整合版 ApplicationSet

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: all-apps
  namespace: argocd
spec:
  generators:
    - matrix:
        generators:
          - git:
              repoURL: https://github.com/org/gitops.git
              revision: HEAD
              directories:
                - path: apps/*
          - list:
              elements:
                - env: dev
                  cluster: https://dev-k8s.example.com
                - env: staging
                  cluster: https://staging-k8s.example.com
                - env: prod
                  cluster: https://prod-k8s.example.com
  strategy:
    type: RollingSync
    rollingSync:
      steps:
        - matchExpressions:
            - key: env
              operator: In
              values: [dev]
        - matchExpressions:
            - key: env
              operator: In
              values: [staging]
        - matchExpressions:
            - key: env
              operator: In
              values: [prod]
  template:
    metadata:
      name: '{{path.basename}}-{{env}}'
      labels:
        app: '{{path.basename}}'
        env: '{{env}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/org/gitops.git
        targetRevision: HEAD
        path: '{{path}}/overlays/{{env}}'
      destination:
        server: '{{cluster}}'
        namespace: '{{path.basename}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

6. ArgoCD CLI 运维命令

# 查看 ApplicationSet 状态
argocd appset list
argocd appset get myapp-progressive

# 查看特定 Application 的 Sync 状态
argocd app get myapp-prod --refresh

# 手动 Sync (Progressive 中需要人工批准时)
argocd app sync myapp-staging

# 查看 Rollout 状态 (Argo Rollouts kubectl plugin)
kubectl argo rollouts status myapp -n myapp
kubectl argo rollouts get rollout myapp -n myapp -w

# Canary 手动批准
kubectl argo rollouts promote myapp -n myapp

# 回滚
kubectl argo rollouts undo myapp -n myapp

7. 最佳实践

  1. Progressive Sync 必备：部署到 prod 时务必分阶段推出
  2. 联动 AnalysisTemplate：配置基于指标的自动回滚
  3. 用 ApplicationSet 代替 App of Apps：大幅降低管理复杂度
  4. 善用 Matrix Generator：自动化应用 × 环境的组合
  5. 善用 Sync Wave：控制有依赖关系的资源顺序
  6. 接入 Notification：用 Slack/Telegram 通知部署状态

本系列接下来该读的文章

8. 小测验

Q1. ApplicationSet 的核心作用是什么？

从一份模板自动生成多个 ArgoCD Application。实现多集群/多环境部署的自动化。

Q2. Matrix Generator 是怎么工作的？

把两个 Generator 的结果做 交叉组合（Cartesian product），为所有组合生成 Application。

Q3. Progressive Sync 中的 maxUpdate: 50% 是什么意思？

在该阶段的 Application 中 只让 50% 同时执行 Sync，成功后再推进剩下的 50%。

Q4. Argo Rollouts 的 AnalysisTemplate 失败后会怎样？

会自动 回滚。超过 failureLimit 后 Rollout 进入 Degraded 状态，并恢复到上一个版本。

Q5. Blue-Green 中 autoPromotionEnabled: false 意味着什么？

必须在 Preview 环境验证之后 手动 promote，才会切换成 Active。需要执行 kubectl argo rollouts promote 命令。

Q6. Canary 发布中为什么要配置 trafficRouting？

为了精确控制实际流量的比例。与 Istio/Nginx/ALB 等联动，实现基于 流量比例 而非 Pod 数量的 Canary。

Q7. 相比 App of Apps 模式，ApplicationSet 的优点是什么？

(1) 用单一资源管理 (2) 通过 Generator 自动发现 (3) 支持 Progressive Sync (4) 消除代码重复

Discuss on Twitter • View on GitHub