In a previous article, we discussed GitOps and its advantages over traditional infrastructure management. We explained how Git is used to store and version this infrastructure code by building a deployment chain to automatically align environments with infrastructure code. In this article, we will discuss the definition of GitOps, its benefits, and some solution ideas for your GitOps projects.
What is GitOps
GitOps is a particular implementation of automated deployment of Cloud Native applications. It aims to build the infrastructure using the same tools used for application development such as Git or CI/CD tools (e.g. Jenkins, GitLab CI).
The concept is to have in a Git repository the declarative description of the desired infrastructure in production and an automated process, which takes care of continuously aligning the desired state of the system with the production environment. So to deploy a new application version or to update it, the Git repository must be updated and the automated process takes care of the rest.
The benefits of GitOps
Which solution should you adopt for your GitOps projects?
On the “infrastructure as code” side, multiple tools are available so that it is not easy to find your way around.
● Cloud Templates (AWS CloudFormation, Azure RM, Google Cloud Deployment Manager),
● Harshicorp Terraform,
● AWS CDK, Troposphere, Palumi
● Kubernetes YAML and DSLs, Knative,
● Chef, Puppet, Ansible, Salt, etc.
At Astrakhan, we particularly appreciate Terraform, which offers more than 500 providers and is therefore compatible with multiple clouds (including Azure, AWS, Google Cloud, Alibaba Cloud, Oracle Cloud) and services (VMWare, Datadog, GitLab, etc.).
The community is very active and maintains a large number of modules to simplify the infrastructure code and its maintenance by hiding the complexity. Another of its advantages is to show the list of changes to be applied for approval before applying them. While this may seem to be in opposition to GitOps and the process of automatic alignment between the desired state of the system and the target environment, it should be understood that this step brings more control with the ability to set up rules dictating whether approval is needed or can be applied automatically.
Like SonarQube for application development, new tools are also emerging to help improve the quality of infrastructure code, such as Open Policy Agent (a framework for writing your own code control policies), Checkov (natively integrates more than 300 controls), and Tf-parliament (natively integrates controls on AWS IAM policies).
Regarding deployment automation (CD), one tool particularly stands out for implementing GitOps and fits perfectly into the idea of using the same tools for development and operations. We are talking about GitLab, which natively offers all the necessary functionalities for the deployment with Terraform:
● Terraform relies on a backend to store the state of the resources it has itself provisioned and its unique identifier. This is the Terraform State. In addition, Terraform Cloud offers a service that allows you to store your Terraform State by creating an account. It is also possible to use services like AWS S3 or Google Cloud Storage. Terraform State management is also offered natively by GitLab and is available directly without requiring any prior action or resources.
● Then there’s GitLab CI, which needs no introduction. Pioneer in terms of “Pipeline as Code”, it was followed by Bitbucket Pipelines, Azure DevOps, or even GitHub Action. GitLab CI now covers the needs of CD by offering integration with Terraform to bring up Terraform reports in your merge requests.
● Something less known is the ease with which you can take advantage of a “deploy board” by simply adding the keyword “environment” in your pipeline code at the job level responsible for deployment (terraform apply command). The deploy board gives you access to the deployment history for each environment and component. Author, date, job and pipeline, commit responsible for each deployment, everything can be found there. As a bonus, it is even possible to re-run a past deployment to go back to an earlier version.
● To keep one’s environment aligned with the desired system state description in Git, a “Scheduled Pipeline” can be set up to launch a pipeline at regular intervals, which takes care of keeping the target environment in sync.
● Notifications are extremely configurable and allow you to be alerted in the event of a deployment incident by email and/or other services.
● GitLab offers integration with a second product from Harshicorp. This is an integration with Vault to externalize and secure access to secrets by defining conditions to read secrets based on information sent by GitLab in the form of a JWT token and verified by rules to be implemented on the Vault side.
● Perhaps your infrastructure projects are complex, broken down into multiple layers, or cross-cutting to many other projects, and you need an equally large pipeline? The “Dynamic Child Pipeline” feature allows you to generate the GitLab CI code to run for each of your layers or projects in a first job. A great idea is to hijack the “Deploy Board” already mentioned to display both deployments and changes pending approval.
Sources:
– https://www.gitops.tech/
– https://www.terraform.io/
– https://www.openpolicyagent.org/docs/latest/terraform/
– https://github.com/bridgecrewio/checkov
– https://github.com/rdkls/tf-parliament
– https://gitlab.com/gitops-demo/readme
– https://docs.gitlab.com/ee/ci/environments/#defining-environments
– https://docs.gitlab.com/ee/user/infrastructure/terraform_state.html#get-started-using-gitlab-ci
– https://docs.gitlab.com/ee/ci/yaml/
– https://docs.gitlab.com/ee/user/project/deploy_boards.html
– https://docs.gitlab.com/ee/ci/yaml/#environment
– https://docs.gitlab.com/ee/ci/pipelines/schedules.html
– https://docs.gitlab.com/ee/user/profile/notifications.html
– https://docs.gitlab.com/ee/user/project/integrations/microsoft_teams.html
– https://docs.gitlab.com/ee/ci/secrets/
– https://docs.gitlab.com/ee/ci/parent_child_pipelines.html
