Trade Study: GitOps
- Conducted by: Tess and Jeff
- Backlog Work Item: #21672
- Decision Makers: Wallace, whole team
Overview
For Memory, we will be creating a cloud native application with infrastructure as code. We will use GitOps for Continuous Deployment through pull requests infrastructure changes to be reflected.
Overall, between our two options, one is more simple and targeted in a way that we believe would meet the requirements for this project. The other does the same, with additional features that may or may not be worth the extra configuration and setup.
Evaluation Criteria
- Repo style: mono versus multi
- Policy Enforcement
- Deployment Methods
- Deployment Monitoring
- Admission Control
- Azure Documentation availability
- Maintainability
- Maturity
- User Interface
Solutions
Flux
Flux is a tool created by Waveworks and is built on top of Kubernetes' API extension system, supports multi-tenancy, and integrates seamlessly with popular tools like Prometheus.
Flux Acceptance Criteria Evaluation
- Repo style: mono versus multi
- Flux supports both as of v2
- Policy Enforcement
- Deployment Methods
- Define a Helm release using Helm Controllers
- Kustomization describes deployments
- Deployment Monitoring
- Flux works with Prometheus for deployment monitoring as well as Grafana dashboards
- Admission Control
- Flux uses RBAC from Kubernetes to lock down sync permissions.
- Uses the service account to access image pull secrets
- Azure Documentation availability
- Great, better when using Helm Operators
- Maintainability
- Manage via YAML files in git repo
- Maturity
- v2 is published under Apache license in GitHub, it works with Helm v3, and has PR commits from as recently as today
- 945 stars, 94 forks
- User Interface
- CLI, the simplest lightweight option
Other features to call out (see more on website)
- Flux only supports Pull-based deployments which means it must be paired with an operator
- Flux can send notifications and receive webhooks for syncing
- Health assessments
- Dependency management
- Automatic deployment
- Garbage collection
- Deploy on commit
Variations
Controllers
Both Controller options are optional.
The Helm Controller additionally fetches helm artifacts to publish, see below diagram.
The Kustomize Controller manages state and continuous deployment.
We will not decide between the controller to use here, as that's a separate trade study, however we will note that Helm is more widely documented within Flux documentation.
Flux v1
Flux v1 is only in maintenance mode and should not be used anymore. So this section does not consider the v1 option a valid option.
GitOps Toolkit
Flux v2 is built on top of the GitOps Toolkit, however we do not evaluate using the GitOps Toolkit alone as that is for when you want to make your own CD system, which is not what we want.
ArgoCD with Helm Charts
ArgoCD is a declarative, GitOps-based Continuous Delivery (CD) tool for Kubernetes.
ArgoCD with Helm Acceptance Criteria Evaluation
- Repo style: mono versus multi
- ArgoCD supports both
- Policy Enforcement
- Deployment Methods
- Deployment Monitoring
- Argo CD expose two sets of Prometheus metrics (application metrics and API server metrics) for deployment monitoring.
- Admission Control
- ArgoCD use RBAC feature. RBAC requires SSO configuration or one or more local users setup. Once SSO or local users are configured, additional RBAC roles can be defined
- Argo CD does not have its own user management system and has only one built-in user admin. The admin user is a superuser, and it has unrestricted access to the system
- Authorization is handled via JWT tokens and checking group claims in them
- Azure Documentation availability
- Argo has documentation on Azure AD
- Maturity
- Has PR commits from as recently as today
- 5,000 stars, 1,100 forks
- Maintainability
- Can use GitOps to manage it
- User Interface
- ArgoCD has a GUI and can be used across clusters
Other features to call out (see more on website)
- ArgoCD support both pull model and push model for continuous delivery
- Argo can send notifications, but you need a separate tool for it
- Argo can receive webhooks
- Health assessments
- Potentially much more useful multi-tenancy tools. Manages multiple projects, maps them to teams, etc.
- SSO Integration
- Garbage collection
Results
This section should contain a table that has each solution rated against each of the evaluation criteria:
| Solution | Repo style | Policy Enforcement | Deployment Methods | Deployment Monitoring | Admission Control | Azure Doc | Maintainability | Maturity | UI |
|---|---|---|---|---|---|---|---|---|---|
| Flux | mono, multi | Azure Policy, preview | Helm, Kustomize | Prometheus, Grafana | RBAC | Yes on Azure | YAML in git repo | 945 stars, 94 forks, currently maintained | CLI |
| ArgoCD | mono, multi | Azure Policy, preview | Helm, Kustomize, KSonnet, ... | Prometheus, Grafana | RBAC | Only in their own docs | manifests in git repo | 5,000 stars, 1,100 forks | GUI, multiple clusters in same GUI |
Decision
ArgoCD is more feature rich, will support more scenarios, and will be a better tool to put in our tool belts. So we have decided at this point to go with ArgoCD.