- Published on
GitOps 완전 가이드 2025: ArgoCD vs Flux, ApplicationSet, Image Updater, 멀티 클러스터
- Authors

- Name
- Youngju Kim
- @fjvbn20031
목차
1. 들어가며: GitOps란 무엇인가
GitOps는 Git 저장소를 인프라와 애플리케이션의 단일 진실의 원천(Single Source of Truth)으로 사용하는 운영 방법론이다. 선언적 설정 파일을 Git에 저장하고, 자동화된 프로세스가 클러스터의 실제 상태를 Git의 원하는 상태와 동기화한다.
GitOps의 4가지 원칙:
- 선언적(Declarative) - 시스템의 원하는 상태를 선언적으로 기술
- 버전 관리(Versioned) - 원하는 상태가 Git에 저장되어 완전한 이력 추적 가능
- 자동 적용(Pulled) - 에이전트가 원하는 상태 변경을 자동으로 감지하고 적용 (Pull 기반)
- 지속 조정(Continuously Reconciled) - 실제 상태와 원하는 상태의 차이를 지속적으로 감지하고 수정
2. ArgoCD 심화
2.1 아키텍처
ArgoCD는 다음 핵심 컴포넌트로 구성된다:
- API Server: UI, CLI, CI/CD 시스템과의 인터페이스
- Repository Server: Git 리포지토리의 매니페스트를 생성하는 내부 서비스
- Application Controller: 실행 중인 애플리케이션 상태를 모니터링하고 동기화
# ArgoCD 설치
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
# CLI 로그인
argocd login argocd.example.com --grpc-web
# 클러스터 등록
argocd cluster add production-context --name production
2.2 Application CRD
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: api-server
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: production
source:
repoURL: https://github.com/myorg/k8s-manifests.git
targetRevision: main
path: apps/api-server/overlays/production
kustomize:
namePrefix: prod-
commonLabels:
env: production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- PrunePropagationPolicy=foreground
- PruneLast=true
- ServerSideApply=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
ignoreDifferences:
- group: apps
kind: Deployment
jsonPointers:
- /spec/replicas
- group: autoscaling
kind: HorizontalPodAutoscaler
jqPathExpressions:
- .spec.metrics[].resource.target
2.3 Sync Hooks
# Pre-sync: DB 마이그레이션
apiVersion: batch/v1
kind: Job
metadata:
name: db-migration
annotations:
argocd.argoproj.io/hook: PreSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
template:
spec:
containers:
- name: migrate
image: myapp/db-migrate:latest
command: ["./migrate", "up"]
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: db-credentials
key: url
restartPolicy: Never
backoffLimit: 3
---
# Post-sync: Slack 알림
apiVersion: batch/v1
kind: Job
metadata:
name: notify-deploy
annotations:
argocd.argoproj.io/hook: PostSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
template:
spec:
containers:
- name: notify
image: curlimages/curl
command:
- /bin/sh
- -c
- |
curl -X POST -H 'Content-type: application/json' \
--data '{"text":"API Server deployed successfully to production"}' \
https://hooks.slack.com/services/T00/B00/xxx
restartPolicy: Never
---
# SyncFail: 실패 시 알림
apiVersion: batch/v1
kind: Job
metadata:
name: notify-fail
annotations:
argocd.argoproj.io/hook: SyncFail
argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
template:
spec:
containers:
- name: notify
image: curlimages/curl
command:
- /bin/sh
- -c
- |
curl -X POST -H 'Content-type: application/json' \
--data '{"text":"ALERT: API Server deployment FAILED in production"}' \
https://hooks.slack.com/services/T00/B00/xxx
restartPolicy: Never
2.4 Health Checks 커스터마이징
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
namespace: argocd
data:
resource.customizations.health.argoproj.io_Rollout: |
hs = {}
if obj.status ~= nil then
if obj.status.currentStepIndex ~= nil then
hs.status = "Progressing"
hs.message = "Rollout in progress"
end
if obj.status.phase == "Healthy" then
hs.status = "Healthy"
hs.message = "Rollout is healthy"
end
if obj.status.phase == "Degraded" then
hs.status = "Degraded"
hs.message = "Rollout is degraded"
end
end
return hs
resource.customizations.health.cert-manager.io_Certificate: |
hs = {}
if obj.status ~= nil then
if obj.status.conditions ~= nil then
for i, condition in ipairs(obj.status.conditions) do
if condition.type == "Ready" and condition.status == "False" then
hs.status = "Degraded"
hs.message = condition.message
return hs
end
if condition.type == "Ready" and condition.status == "True" then
hs.status = "Healthy"
hs.message = condition.message
return hs
end
end
end
end
hs.status = "Progressing"
hs.message = "Waiting for certificate"
return hs
2.5 RBAC 설정
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-rbac-cm
namespace: argocd
data:
policy.default: role:readonly
policy.csv: |
# 프로젝트 관리자
p, role:project-admin, applications, *, production/*, allow
p, role:project-admin, applications, sync, production/*, allow
p, role:project-admin, logs, get, production/*, allow
# 개발자: 읽기 + sync만
p, role:developer, applications, get, */*, allow
p, role:developer, applications, sync, staging/*, allow
p, role:developer, logs, get, */*, allow
# CI/CD 시스템
p, role:ci-cd, applications, sync, */*, allow
p, role:ci-cd, applications, get, */*, allow
# 그룹 매핑
g, platform-team, role:admin
g, backend-team, role:project-admin
g, frontend-team, role:developer
g, github-actions, role:ci-cd
scopes: '[groups, email]'
3. Flux 심화
3.1 핵심 리소스
# GitRepository: Git 소스 정의
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: platform-repo
namespace: flux-system
spec:
interval: 1m
url: https://github.com/myorg/platform-config
ref:
branch: main
secretRef:
name: git-credentials
ignore: |
# exclude all
/*
# include deploy dir
!/deploy
---
# Kustomization: 무엇을 어디에 배포할지 정의
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: api-server
namespace: flux-system
spec:
interval: 5m
retryInterval: 2m
timeout: 3m
sourceRef:
kind: GitRepository
name: platform-repo
path: ./deploy/apps/api-server/production
prune: true
wait: true
healthChecks:
- apiVersion: apps/v1
kind: Deployment
name: api-server
namespace: production
patches:
- patch: |
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
spec:
replicas: 5
target:
kind: Deployment
name: api-server
---
# HelmRelease: Helm 차트 배포
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: redis
namespace: production
spec:
interval: 10m
chart:
spec:
chart: redis
version: "18.x"
sourceRef:
kind: HelmRepository
name: bitnami
namespace: flux-system
values:
architecture: replication
auth:
enabled: true
existingSecret: redis-auth
replica:
replicaCount: 3
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: "1"
memory: 1Gi
valuesFrom:
- kind: ConfigMap
name: redis-values
valuesKey: values.yaml
upgrade:
remediation:
retries: 3
rollback:
cleanupOnFail: true
3.2 Flux 알림 설정
# Provider: Slack 알림
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
name: slack
namespace: flux-system
spec:
type: slack
channel: deployments
secretRef:
name: slack-webhook
---
# Alert: 특정 이벤트에 대한 알림
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
name: deployment-alerts
namespace: flux-system
spec:
providerRef:
name: slack
eventSeverity: error
eventSources:
- kind: Kustomization
name: "*"
- kind: HelmRelease
name: "*"
exclusionList:
- ".*upgrade.*retries exhausted.*"
summary: "Flux deployment issue detected"
4. ArgoCD vs Flux 비교
| 항목 | ArgoCD | Flux |
|---|---|---|
| UI | 풍부한 웹 UI 내장 | 별도 UI 필요 (Weave GitOps 등) |
| 아키텍처 | 중앙 집중형 | 분산형, 네임스페이스 단위 |
| CRD 수 | 적음 (Application, AppProject) | 많음 (GitRepo, Kustomization, HelmRelease 등) |
| Helm 지원 | Template 렌더링 후 적용 | Native HelmRelease CRD |
| Kustomize | 내장 지원 | 내장 지원 |
| 멀티테넌시 | AppProject로 격리 | 네임스페이스 기반 격리 |
| Image 자동 업데이트 | ArgoCD Image Updater | Flux Image Automation |
| SSO | OIDC/SAML/LDAP 지원 | UI 도구에 따라 다름 |
| RBAC | 세밀한 역할 기반 접근 제어 | Kubernetes RBAC 활용 |
| CLI | argocd CLI | flux CLI |
| 알림 | Notification 설정 | Alert/Provider CRD |
| Health Check | 커스텀 Lua 스크립트 | Kustomization 헬스 체크 |
| Drift 감지 | 실시간 UI 표시 | 주기적 Reconciliation |
| Progressive Delivery | Argo Rollouts 연동 | Flagger 연동 |
| 러닝 커브 | 중간 (UI가 도움) | 높음 (CRD 기반 설정) |
5. ApplicationSet
5.1 List Generator
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: microservices
namespace: argocd
spec:
generators:
- list:
elements:
- cluster: production
url: https://prod-k8s.example.com
namespace: production
replicas: "5"
- cluster: staging
url: https://staging-k8s.example.com
namespace: staging
replicas: "2"
- cluster: development
url: https://dev-k8s.example.com
namespace: development
replicas: "1"
template:
metadata:
name: "api-server-{{cluster}}"
spec:
project: default
source:
repoURL: https://github.com/myorg/k8s-manifests.git
targetRevision: main
path: "apps/api-server/overlays/{{cluster}}"
kustomize:
commonLabels:
cluster: "{{cluster}}"
destination:
server: "{{url}}"
namespace: "{{namespace}}"
syncPolicy:
automated:
prune: true
selfHeal: true
5.2 Cluster Generator
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: platform-monitoring
namespace: argocd
spec:
generators:
- clusters:
selector:
matchLabels:
monitoring: enabled
values:
prometheus_retention: "30d"
template:
metadata:
name: "monitoring-{{name}}"
spec:
project: platform
source:
repoURL: https://github.com/myorg/platform-charts.git
targetRevision: main
path: charts/monitoring
helm:
values: |
cluster: "{{name}}"
prometheus:
retention: "{{values.prometheus_retention}}"
grafana:
ingress:
host: "grafana-{{name}}.example.com"
destination:
server: "{{server}}"
namespace: monitoring
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
5.3 Git Generator
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: team-apps
namespace: argocd
spec:
generators:
- git:
repoURL: https://github.com/myorg/k8s-manifests.git
revision: main
directories:
- path: "apps/*/overlays/production"
- path: "apps/legacy-*"
exclude: true
template:
metadata:
name: "{{path[1]}}"
spec:
project: default
source:
repoURL: https://github.com/myorg/k8s-manifests.git
targetRevision: main
path: "{{path}}"
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
---
# Git File Generator: config.json 파일 기반
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: dynamic-apps
namespace: argocd
spec:
generators:
- git:
repoURL: https://github.com/myorg/app-config.git
revision: main
files:
- path: "environments/*/config.json"
template:
metadata:
name: "{{app.name}}-{{environment}}"
spec:
project: "{{app.project}}"
source:
repoURL: "{{app.repo}}"
targetRevision: "{{app.revision}}"
path: "{{app.path}}"
destination:
server: "{{cluster.url}}"
namespace: "{{app.namespace}}"
5.4 Matrix Generator
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: cross-cluster-apps
namespace: argocd
spec:
generators:
- matrix:
generators:
- clusters:
selector:
matchLabels:
env: production
- list:
elements:
- app: api-server
chart: api-server
namespace: backend
- app: web-frontend
chart: web-frontend
namespace: frontend
- app: worker
chart: worker
namespace: backend
template:
metadata:
name: "{{app}}-{{name}}"
spec:
project: production
source:
repoURL: https://github.com/myorg/helm-charts.git
targetRevision: main
path: "charts/{{chart}}"
helm:
valueFiles:
- "values-{{metadata.labels.region}}.yaml"
destination:
server: "{{server}}"
namespace: "{{namespace}}"
syncPolicy:
automated:
prune: true
selfHeal: true
5.5 Pull Request Generator
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: pr-preview
namespace: argocd
spec:
generators:
- pullRequest:
github:
owner: myorg
repo: api-server
tokenRef:
secretName: github-token
key: token
labels:
- preview
requeueAfterSeconds: 30
template:
metadata:
name: "pr-{{number}}-api-server"
labels:
preview: "true"
spec:
project: previews
source:
repoURL: https://github.com/myorg/api-server.git
targetRevision: "{{head_sha}}"
path: deploy/preview
kustomize:
namePrefix: "pr-{{number}}-"
commonLabels:
pr: "{{number}}"
destination:
server: https://kubernetes.default.svc
namespace: "preview-{{number}}"
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
6. Image Updater
6.1 ArgoCD Image Updater
# 설치
# kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj-labs/argocd-image-updater/stable/manifests/install.yaml
# Application에 이미지 업데이트 설정
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: api-server
namespace: argocd
annotations:
# 이미지 목록과 업데이트 전략
argocd-image-updater.argoproj.io/image-list: "app=myregistry.com/api-server"
argocd-image-updater.argoproj.io/app.update-strategy: semver
argocd-image-updater.argoproj.io/app.allow-tags: "regexp:^v[0-9]+\\.[0-9]+\\.[0-9]+$"
argocd-image-updater.argoproj.io/app.ignore-tags: "latest, nightly"
# Git write-back 방식
argocd-image-updater.argoproj.io/write-back-method: git
argocd-image-updater.argoproj.io/write-back-target: "kustomization:../../base"
argocd-image-updater.argoproj.io/git-branch: main
spec:
project: production
source:
repoURL: https://github.com/myorg/k8s-manifests.git
targetRevision: main
path: apps/api-server/overlays/production
destination:
server: https://kubernetes.default.svc
namespace: production
6.2 Flux Image Automation
# ImageRepository: 이미지 레지스트리 감시
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
name: api-server
namespace: flux-system
spec:
image: myregistry.com/api-server
interval: 5m
secretRef:
name: registry-credentials
---
# ImagePolicy: 어떤 태그를 선택할지 정의
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
name: api-server
namespace: flux-system
spec:
imageRepositoryRef:
name: api-server
policy:
semver:
range: ">=1.0.0"
filterTags:
pattern: "^v(?P<version>[0-9]+\\.[0-9]+\\.[0-9]+)$"
extract: "$version"
---
# ImageUpdateAutomation: Git 자동 커밋
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
name: auto-update
namespace: flux-system
spec:
interval: 5m
sourceRef:
kind: GitRepository
name: platform-repo
git:
checkout:
ref:
branch: main
commit:
author:
name: flux-bot
email: flux@myorg.com
messageTemplate: |
chore(image): update images
Automated image update:
{{range .Changed.Changes}}
- {{.OldValue}} -> {{.NewValue}}
{{end}}
push:
branch: main
update:
path: ./deploy
strategy: Setters
7. Helm + GitOps
7.1 ArgoCD에서 Helm 사용
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nginx-ingress
namespace: argocd
spec:
project: platform
source:
repoURL: https://kubernetes.github.io/ingress-nginx
chart: ingress-nginx
targetRevision: 4.9.1
helm:
releaseName: nginx-ingress
values: |
controller:
replicaCount: 3
resources:
requests:
cpu: 200m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
metrics:
enabled: true
serviceMonitor:
enabled: true
admissionWebhooks:
enabled: true
valueFiles:
- values-production.yaml
parameters:
- name: controller.service.type
value: LoadBalancer
destination:
server: https://kubernetes.default.svc
namespace: ingress-system
syncPolicy:
automated:
prune: true
syncOptions:
- CreateNamespace=true
7.2 Umbrella Chart 패턴
# Chart.yaml (umbrella chart)
apiVersion: v2
name: platform-stack
version: 1.0.0
dependencies:
- name: prometheus
version: 25.x.x
repository: https://prometheus-community.github.io/helm-charts
condition: prometheus.enabled
- name: grafana
version: 7.x.x
repository: https://grafana.github.io/helm-charts
condition: grafana.enabled
- name: loki
version: 5.x.x
repository: https://grafana.github.io/helm-charts
condition: loki.enabled
8. Kustomize + GitOps
8.1 환경별 Overlays
apps/api-server/
base/
deployment.yaml
service.yaml
kustomization.yaml
overlays/
development/
kustomization.yaml
patches/
replicas.yaml
staging/
kustomization.yaml
patches/
replicas.yaml
resources.yaml
production/
kustomization.yaml
patches/
replicas.yaml
resources.yaml
hpa.yaml
# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml
commonLabels:
app: api-server
---
# overlays/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
namePrefix: prod-
patches:
- path: patches/replicas.yaml
- path: patches/resources.yaml
- path: patches/hpa.yaml
configMapGenerator:
- name: api-config
literals:
- LOG_LEVEL=warn
- ENVIRONMENT=production
---
# overlays/production/patches/replicas.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
spec:
replicas: 5
9. 시크릿 관리
9.1 Sealed Secrets
# 설치
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets sealed-secrets/sealed-secrets -n kube-system
# 시크릿 암호화
kubectl create secret generic db-credentials \
--from-literal=username=admin \
--from-literal=password=super-secret \
--dry-run=client -o yaml | \
kubeseal --controller-name=sealed-secrets \
--controller-namespace=kube-system \
--format yaml > sealed-db-credentials.yaml
# sealed-db-credentials.yaml (Git에 커밋 가능)
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: db-credentials
namespace: production
spec:
encryptedData:
username: AgB1k2j3l4m5n6o7p8...
password: AgC9d0e1f2g3h4i5j6...
template:
metadata:
name: db-credentials
namespace: production
type: Opaque
9.2 External Secrets Operator
# SecretStore: Vault 연결
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: vault-backend
namespace: production
spec:
provider:
vault:
server: https://vault.example.com
path: secret
version: v2
auth:
kubernetes:
mountPath: kubernetes
role: production-app
serviceAccountRef:
name: vault-auth
---
# ExternalSecret: Vault에서 시크릿 동기화
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: db-credentials
namespace: production
spec:
refreshInterval: 1h
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: db-credentials
creationPolicy: Owner
template:
type: Opaque
data:
DATABASE_URL: "postgresql://{{ .username }}:{{ .password }}@db.production:5432/myapp"
data:
- secretKey: username
remoteRef:
key: production/database
property: username
- secretKey: password
remoteRef:
key: production/database
property: password
---
# ClusterSecretStore: 클러스터 전체에서 사용
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
name: aws-secrets-manager
spec:
provider:
aws:
service: SecretsManager
region: ap-northeast-2
auth:
jwt:
serviceAccountRef:
name: external-secrets-sa
namespace: external-secrets
9.3 SOPS (Secrets OPerationS)
# .sops.yaml 설정 파일
creation_rules:
- path_regex: ".*\\.enc\\.yaml$"
encrypted_regex: "^(data|stringData)$"
age: age1ql3z7hjy54pw3hyww5ay...
- path_regex: "environments/production/.*"
kms: "arn:aws:kms:ap-northeast-2:123456789012:key/abc-123"
- path_regex: "environments/staging/.*"
pgp: "FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4"
# SOPS로 암호화
sops --encrypt --in-place secrets.enc.yaml
# Flux에서 SOPS 사용을 위한 Kustomization
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: api-server
namespace: flux-system
spec:
interval: 5m
sourceRef:
kind: GitRepository
name: platform-repo
path: ./deploy/apps/api-server
prune: true
decryption:
provider: sops
secretRef:
name: sops-age-key
10. Progressive Delivery
10.1 Argo Rollouts - 카나리 배포
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: api-server
namespace: production
spec:
replicas: 10
revisionHistoryLimit: 3
selector:
matchLabels:
app: api-server
template:
metadata:
labels:
app: api-server
spec:
containers:
- name: api-server
image: myregistry.com/api-server:v2.0.0
ports:
- containerPort: 8080
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: "1"
memory: 1Gi
strategy:
canary:
canaryService: api-server-canary
stableService: api-server-stable
trafficRouting:
istio:
virtualServices:
- name: api-server-vsvc
routes:
- primary
steps:
- setWeight: 5
- pause:
duration: 5m
- setWeight: 20
- pause:
duration: 10m
- setWeight: 50
- pause:
duration: 10m
- setWeight: 80
- pause:
duration: 5m
analysis:
templates:
- templateName: success-rate
- templateName: latency
startingStep: 1
args:
- name: service-name
value: api-server-canary
---
# AnalysisTemplate: 성공률 검증
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
spec:
args:
- name: service-name
metrics:
- name: success-rate
interval: 60s
successCondition: result[0] >= 0.99
failureLimit: 3
provider:
prometheus:
address: http://prometheus.monitoring:9090
query: |
sum(rate(http_requests_total{service="{{args.service-name}}",status=~"2.."}[5m]))
/
sum(rate(http_requests_total{service="{{args.service-name}}"}[5m]))
---
# AnalysisTemplate: 지연 시간 검증
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: latency
spec:
args:
- name: service-name
metrics:
- name: p99-latency
interval: 60s
successCondition: result[0] < 500
failureLimit: 3
provider:
prometheus:
address: http://prometheus.monitoring:9090
query: |
histogram_quantile(0.99,
sum(rate(http_request_duration_seconds_bucket{service="{{args.service-name}}"}[5m]))
by (le)
) * 1000
10.2 Argo Rollouts - 블루그린 배포
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: web-frontend
namespace: production
spec:
replicas: 5
selector:
matchLabels:
app: web-frontend
template:
metadata:
labels:
app: web-frontend
spec:
containers:
- name: web
image: myregistry.com/web-frontend:v3.0.0
ports:
- containerPort: 3000
strategy:
blueGreen:
activeService: web-frontend-active
previewService: web-frontend-preview
autoPromotionEnabled: false
scaleDownDelaySeconds: 300
prePromotionAnalysis:
templates:
- templateName: smoke-tests
args:
- name: preview-url
value: "http://web-frontend-preview.production.svc:3000"
postPromotionAnalysis:
templates:
- templateName: success-rate
args:
- name: service-name
value: web-frontend-active
11. 멀티 클러스터 GitOps
11.1 Hub-Spoke 패턴
management-cluster/ (Hub)
argocd/
applicationsets/
monitoring.yaml -> 모든 클러스터에 모니터링 배포
logging.yaml -> 모든 클러스터에 로깅 배포
apps.yaml -> 환경별 앱 배포
projects/
platform.yaml
production.yaml
staging.yaml
clusters/
prod-us-east.yaml
prod-ap-northeast.yaml
staging-us-east.yaml
11.2 저장소 구조: Monorepo vs Polyrepo
Monorepo 구조:
k8s-manifests/
apps/
api-server/
base/
overlays/
development/
staging/
production/
web-frontend/
base/
overlays/
development/
staging/
production/
platform/
monitoring/
logging/
ingress/
clusters/
production/
staging/
Polyrepo 구조:
# repo: api-server-deploy
deploy/
base/
overlays/
development/
staging/
production/
# repo: web-frontend-deploy
deploy/
base/
overlays/
development/
staging/
production/
# repo: platform-config
monitoring/
logging/
ingress/
| 항목 | Monorepo | Polyrepo |
|---|---|---|
| 가시성 | 전체 시스템 한눈에 | 서비스별 독립적 |
| 접근 제어 | 디렉토리 기반 (CODEOWNERS) | 리포지토리별 |
| CI/CD | 변경 감지 복잡 | 단순 트리거 |
| 의존성 관리 | 쉬움 | 버전 관리 필요 |
| 규모 | 중소 규모 적합 | 대규모 조직 적합 |
11.3 App-of-Apps 패턴
# 루트 Application: 다른 Application들을 관리
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: root-app
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/myorg/k8s-manifests.git
targetRevision: main
path: clusters/production
destination:
server: https://kubernetes.default.svc
namespace: argocd
syncPolicy:
automated:
prune: true
selfHeal: true
12. Drift 감지와 Reconciliation
12.1 ArgoCD Drift 감지
ArgoCD는 실시간으로 Git의 원하는 상태와 클러스터의 실제 상태를 비교한다.
# 특정 필드를 Drift 감지에서 제외
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: api-server
spec:
ignoreDifferences:
# HPA가 관리하는 replicas 무시
- group: apps
kind: Deployment
jsonPointers:
- /spec/replicas
# 자동 생성되는 annotation 무시
- group: ""
kind: Service
jqPathExpressions:
- .metadata.annotations."service.beta.kubernetes.io/aws-load-balancer-*"
# MutatingWebhook이 주입하는 sidecar 무시
- group: apps
kind: Deployment
managedFieldsManagers:
- istio-sidecar-injector
12.2 Flux Reconciliation
# 강제 Reconciliation
# flux reconcile kustomization api-server --with-source
# Reconciliation 상태 확인
# flux get kustomizations
# flux get helmreleases -A
# 특정 리소스 일시 중단
# flux suspend kustomization api-server
# flux resume kustomization api-server
13. 실전 퀴즈
Q1: GitOps의 Pull 기반 모델이 Push 기반(전통적 CI/CD)보다 보안적으로 우수한 이유는?
정답: Pull 기반 모델에서는 클러스터 내부의 에이전트(ArgoCD/Flux)가 Git을 폴링하여 변경 사항을 적용한다. CI 파이프라인에 클러스터 접근 자격 증명을 노출할 필요가 없다.
- Push 모델: CI 서버가 kubectl/helm으로 직접 배포 -> 클러스터 자격 증명이 CI에 필요
- Pull 모델: 클러스터 내부 에이전트가 Git을 감시 -> 클러스터 자격 증명이 외부에 노출되지 않음
- 공격 표면(Attack Surface) 감소: CI 시스템이 침해되어도 클러스터에 직접 접근 불가
Q2: ApplicationSet의 Matrix Generator는 어떤 상황에서 유용한가?
정답: Matrix Generator는 두 개의 생성기를 조합하여 카르테시안 곱(모든 조합)을 만든다. 여러 클러스터에 여러 애플리케이션을 배포할 때 유용하다.
- 예: 3개 클러스터(prod-us, prod-eu, prod-ap) x 5개 앱 = 15개 Application 자동 생성
- Cluster Generator + List Generator 조합이 가장 일반적
- 중복 설정 없이 모든 조합을 자동으로 관리
Q3: ArgoCD Image Updater의 write-back-method가 git인 경우와 argocd인 경우의 차이는?
정답: git 방식은 이미지 태그 변경을 Git 리포지토리에 커밋한다. argocd 방식은 ArgoCD Application의 파라미터를 직접 수정한다.
- git: Git이 단일 진실의 원천으로 유지됨. 감사 추적 가능. 속도는 느림
- argocd: Git을 수정하지 않음. 빠르지만 Git과 실제 상태가 불일치할 수 있음
- 프로덕션에서는 git 방식을 권장
Q4: Sealed Secrets vs External Secrets Operator의 주요 차이점은?
정답: Sealed Secrets는 시크릿을 암호화하여 Git에 저장하고, External Secrets Operator는 외부 시크릿 저장소(Vault, AWS SM 등)에서 런타임에 동기화한다.
- Sealed Secrets: Git에 암호화된 시크릿 저장. 오프라인에서도 작동. 키 로테이션이 복잡
- External Secrets Operator: 외부 저장소에서 실시간 동기화. 중앙 집중 관리. 외부 서비스 의존성
- 소규모 팀: Sealed Secrets가 더 간단
- 대규모 조직: External Secrets + Vault 조합 권장
Q5: Argo Rollouts의 카나리 배포에서 AnalysisTemplate이 실패하면 어떤 일이 발생하는가?
정답: AnalysisTemplate의 메트릭이 successCondition을 만족하지 못하고 failureLimit에 도달하면, Rollout이 자동으로 롤백된다.
- 카나리 트래픽이 0%로 되돌아감
- stable 버전이 모든 트래픽을 받음
- Rollout 상태가 Degraded로 변경
- 수동 재시도: kubectl argo rollouts retry rollout api-server
- 분석 결과 확인: kubectl argo rollouts get rollout api-server