Flux vs ArgoCD
I have been trying to find any comparison between FluxCD and ArgoCD that could help me choose between them. I haven’t found any that truly help us at Playtomic (many were just a list of meaningless comments if you don’t know both systems). So… we tried both and made our own decision.
Goals
We are adapting the GitOps practices. After digging into the topic, our ideal setup is:
- The cluster (infrastructure + applications) has to be described as code, not as commands.
- That makes the cluster easy to replicate (eases the process to set up new environments or the disaster recovery of the existing ones).
- As few steps to build a new cluster as possible (how many actions we have to perform manually before the cluster starts running). Spoiler: you can make it with 2 commands: one to build the K8s cluster, another to install all your infra and apps.
- The cluster must monitor changes in our repositories and automatically sync them to the cluster.
- One repository to describe the infrastructure (cluster repo).
- One repository to every service/application we want to deploy in the cluster (applications repo).
These two last items are called multi-tenancy. The goal is that every team can be responsible for the lifecycle of their applications: they can write anything in their repositories, but they cannot alter the cluster infrastructure by themselves. That’s the responsibility of the owners of the cluster repo.
Long story short: you can do everything in that list with both Flux and ArgoCD but there are differences about how they attain that.
Flux vs ArgoCD
Concepts
Flux:
- You have to define GitRepositories/HelmRepositories and make Flux monitor them (if you are using Git+Kustomize you will need a Kustomization)
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
name: service-one
namespace: playtomic
spec:
ref:
branch: develop
interval: 1m
url: https://github.com/your-org/your-repo
secretRef:
name: git-credentials
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
metadata:
name: service-one
namespace: playtomic
spec:
interval: 1m
sourceRef:
kind: GitRepository
name: service
path: deploy/develop
prune: true
ArgoCD
- You have to install Applications. An Application is basically a repo to monitor. It supports Kustomize, Helm, …
- An Application can contain references to more Applications.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: service-one
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: https://github.com/your-org/your-repo
targetRevision: develop
path: deploy/develop
destination:
server: https://kubernetes.default.svc
namespace: playtomic
# Sync policy
syncPolicy:
automated: {}
Main differences
- ArgoCD objects have to be installed in the namespace where ArgoCD is installed (argocd in this example). That is the Application object belongs to the ArgoCD namespace. The actual application can be deployed in any target namespace.
- Flux objects belong to the namespace where the application will be actually deployed.
- ArgoCD supports defining several projects with different permissions (what namespaces or Kubernetes objects the application in that project are allowed to use).
How to start the cluster
Flux:
- You write your cluster using standard k8s objects and Flux objects to indicate what pieces it has to monitor. Dependencies are written as you would write the declarative files in a regular k8s setup.
- You have to run
flux bootstrap
command, and everything is set up and monitored.
ArgoCD:
- You have to install ArgoCD (using the regular k8s descriptors).
- You have to make ArgoCD “self-aware” after that. That is, you have to install ArgoCD as an application in ArgoCD. You will need a secret to let ArgoCD access your repositories.
- Or use ArgoCD Autopilot (which takes care of both steps)
Deploying new versions of your services
You probably have a CI already building your images. You can let that CI commit that version to your repository too (be aware of triggered builds afterward) or you can let Flux/ArgoCD check for new images and do that commit for you.
We wanted to stop our Jenkins CI to deploy actively our services. That’s the CD responsibility and besides, the version deployed was not written anywhere: we had to check our monitor tools or query directly the cluster. So… let’s see what Flux and ArgoCD have to say about this.
Flux
You have to add two extra components when bootstrapping Flux:
flux bootstrap github --owner=your-org --repository=your-cluster-repo --token-auth --path=/clusters/develop --components-extra=image-reflector-controller,image-automation-controller
In every repository where you are storing image versions, you have to add a ImageRepository, a ImagePolicy and a ImageUpdateAutomation.
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageRepository
metadata:
name: service-one
namespace: playtomic
spec:
image: your-docker-prefix/one-service
interval: 1m0s
secretRef:
name: nexus-credentials
---
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImagePolicy
metadata:
name: service-one
namespace: playtomic
spec:
imageRepositoryRef:
name: configuration
filterTags:
pattern: '^develop-(?P<version>.*)'
extract: '$version'
policy:
numerical:
order: asc
---
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageUpdateAutomation
metadata:
name: service-one
namespace: playtomic
spec:
interval: 1m0s
sourceRef:
kind: GitRepository
name: one-service
git:
commit:
author:
email: fluxcdbot@users.noreply.github.com
name: fluxcdbot
messageTemplate: '{{range .Updated.Images}}{{println .}}{{end}}'
update:
path: /deploy/develop/
strategy: Setters
ArgoCD
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: configuration-service
namespace: argocd
annotations:
argocd-image-updater.argoproj.io/image-list: service-one=your-docker-prefix/service-one
argocd-image-updater.argoproj.io/service-one.allow-tags: regexp:^develop-[0-9a-zA-Z\-]+$
argocd-image-updater.argoproj.io/service-one.update-strategy: latest
argocd-image-updater.argoproj.io/write-back-method: git:secret:argocd/github-credentials
argocd-image-updater.argoproj.io/write-back-target: kustomization
argocd-image-updater.argoproj.io/git-branch: develop
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: https://github.com/your-org/your-service
targetRevision: develop
path: deploy/develop
...
You have several strategies, but here we are setting write-back-target: kustomization
so it will write the image versions into the kustomization.yaml
in deploy/develop
directory.
Main differences
- Flux needs two extra components when bootstrapping and then an image-updater per repository whose version is monitoring.
- ArgoCD handles everything in the Application. Flux requires a Kustomization and a GitRepository.
- Flux’s image updater is a component in your tenant repository. ArgoCD’s updater is configured via annotation in their Application object.
- ArgoCD has a web console (with user management). Flux does not. We found that ArgoCD web UI is super useful for not experts to debug the errors in their projects.
- Flux bootstrap command install and make the system monitor itself. ArgoCD does not by default, you will need the ArgoCD autopilot.
Dependencies control
Sometimes you need to install some components in Kubernetes before others. In ArgoCD they managed that dependency order as waves. In Flux, by default, it retries until everything is ready (at least in our experience)
Summary
- In both systems, it feels like the documentation was written expecting that you have already mastered the topic.
- Flux seems more k8s-ish: everything looks like a k8s object. More raw for my taste.
- ArgoCD seems more refined: Applications seem better defined.
- Flux Auto Update is tricky to configure (you have to configure it in two repositories).
- ArgoCD Auto Update seems smoother to set up (it’s a property of the Application).
- ArgoCD has a web console while Flux does not: it makes errors debugging much simpler.
How to decide
If you have time, try both and check what adapts best to your case. I know, you don’t have the time and that’s why you are here.
For us, the decisive points are:
- The image updater configuration is simpler in ArgoCD.
- The image updater in Flux was killing our Nexus with requests: last versions of the updater were unable to query by image name, they have to query everything and then filter the images. As we had several services, one image updater per service, it was too much.
- The ArgoCD console lets our team migrate their services and be able to check their deployments easily.
- Question of taste: ArgoCD descriptors seem more friendly to me.