Continuous Integration (CI) and Continous Delivery/Deployment (CD)
When it comes to a software organization, the primary responsibility of the developers is to implement changes and the primary responsibility of the operations team is to deliver (deploy) the changes to production. This process of sending written code to production involves a set of steps, to ensure proper standards are maintained, the code complies with the security requirements, the changes are versioned efficiently, it is delivered to the suitable clusters and it becomes available to customers with minimal effort and time. Doing this manually every time a change is implemented, could be error-prone, tedious, and time-consuming (often referred to as the integration hell).
In simple terms, CI/CD would be the part right after development to how it gets delivered to a customer. As an accepted definition, CI/CD refers to the automation of these steps, and it is also often referred to as a pipeline. In this article, I would like to explain the basics of CI/CD, why we have it, and how it helps automate the operation side of the software.
Here, I have separated the CD as Delivery and Deployment for clarity. But in general, these are both interchangeably used to represent both terms.
What is CI/CD
CI/CD is the process of introducing automation to frequently deliver application changes to production. The term CI stands for “Continuous Integration” and the term CD stands for both “Continuous Delivery” and/or “Continuous Deployment”.
CI/CD embodies a culture, operating principles, and a set of practices that application development teams use to deliver code changes more frequently and reliably. This allows organizations to ship software quickly and efficiently.
Let’s have a look at what CI and CD mean, as separate terms. I would also like to separately explain the two sides of CD (Delivery/Deployment). It’s important to understand each separation so that the right set of tools and processes can be picked to achieve this.
Continuous Integration (CI)
This is a set of practices that drive development teams to frequently implement small code changes and check them, into a version control repository. In many organizations, code changes are often committed to a version control system (eg: git). With these changes, it is important to collaborate with other changes and the other developers working with the same code base. Once the necessary changes are reviewed, they are merged into a common branch often referred to as the master/main branch. This branch would be the final code, that gets deployed to production.
Therefore, it is important to make sure that the changes being merged to the final branch, are working, reliable, and acceptable. These changes need to be verified, reviewed, and tested before merging into the final branch.
Now imagine having many changes, or even having huge changes, that take a longer time to review and test, whereas frequent changes would make this even harder. This hinders the process of delivering code changes to production. Continuous Integration (CI) tries to solve this problem by automating this entire process.
Of course, there is some work up front to set this up and add the relevant unit and integration testing. But it would make the life of developers and operations teams much easier and reduces the number of human errors that could occur. In general, the following areas are covered in CI
- Building the code
- Testing the code
- Merging the code (to a final branch)
When a developer sets up a pull request (or a merge request), the build process kicks in. Once it is built, the testing will run on top of it. Once it is completed, it would be ready to merge.
See the following high-level setup of CI in your code base.
Some common tools, available out there for you to achieve CI, are:
- Github actions
- Gitlab pipelines
- Jenkins
- Travis CI
- Circle CI
- Etc
Continuous Delivery (CD)
Once the builds are tested and verified in CI, those builds need to move to a repository, artifact store, or registry. A common example would be a Docker Image Registry. The goal of continuous delivery would be to get a build ready for deployment in a suitable format. This is often referred to as a production-ready build.
This also contains versioning, but it is often referred to as releases of the build.
Some common tools, available out there for you to achieve CD (Delivery), are:
- Github actions
- Gitlab pipelines
- Jenkins
- Etc
Next, let’s have a look at what Continuous Deployment means. Please note that Delivery and Deployment are often used interchangeably as CD to represent both.
Continuous Deployment (CD)
This is the final stage of a CI/CD pipeline. This stage is all about releasing a production-ready build to production. In other words, this is about making the changes available to the customer. In the end stage, the developers' efforts will go live at this point.
In most cases, this happens in multiple stages. First, the release happens in a staging environment. With a manual trigger, it gets released to the production environment. When it comes to CD, there are many deployment models out there, that make the releases smooth and seamless. For example, we may want to deploy the changes to a set of Beta customers in production and then roll it out to everyone in production. These deployment models are a topic for another article.
Some common CD tools are:
- Spinnaker
- Go CD
- Argo CD
- Concourse
- Screwdriver
- Etc
CI/CD Fundamentals
For your CI/CD process to be efficient, it is important to include the following fundamentals when building your CI/CD process for each application. There are no hard and fast rules as to what is the right way of implementing your CI/CD pipelines, but this would guide you towards building a more robust set of pipelines for a smoother delivery of applications.
- A single source repository: having a code base for each build would serve the purpose of a repository. This will contain the build files, test scripts, and the actual feature code. Some teams maintain a monolithic repo, doing multiple builds from the same code base. All these drills down to how the system is designed.
- Frequent commits: Having focused and small commits in a VCS repo, would help in quicker deliveries and cleaner changes. Multiple commits to the repository results in fewer places for conflicts to hide. Make small, frequent iterations rather than major changes. By doing this, it’s possible to roll changes back easily if there’s a problem or conflict.
- Automated Builds: This is to support the CI process by having scripts, allowing automated builds when triggered.
- Testable code: This also includes test scripts along with having a code base that is testable. Unit tests and integration tests will help determine issues in the CI phase itself.
- Deployment controls: This refers to the way the code should be deployed in production. Some teams use blue/green deploys, canary deploys, and traffic vectoring to reduce risk during deploys. This can actually take many shapes as people interpret the idea differently to suit their needs
Benefits of CI/CD for your organization
- Satisfied users: Fewer bugs and errors make it into production, so your users and customers have a better experience. This leads to improved levels of customer satisfaction, confidence, and reputation.
- Accelerated time-to-value: When you can deploy anytime, you can bring products and new features to market faster. Your development costs are lower, and a faster turnaround frees your team for other work. Customers get results faster and gain a competitive edge.
- Less fire fighting: Testing code more often, in smaller batches, and earlier in the development cycle can seriously cut down on fire drills. This results in a smoother development cycle and less team stress. Results are more predictable, and it’s easier to find and fix bugs.
- Hit dates more reliably: Removing deployment bottlenecks and making deployments predictable can remove a lot of the uncertainty around hitting key dates. Breaking work into smaller, manageable bites means it’s easier to complete each stage on time and track progress. This approach gives plenty of time to monitor overall progress and determine completion dates more accurately.
- Free up developers’ time: With more of the deployment process automated, the team has time for more rewarding projects. It’s estimated that developers spend between 35% and 50% of their time testing, validating, and debugging code. By automating these processes, developers significantly improve their productivity.
- Less context switching: Getting real-time feedback on the code developers commit makes it easier to work on one thing at a time and minimize cognitive load. By working with small sections of code that are automatically tested, developers can debug code quickly while their minds are still fresh from programming. Finding bugs is easier because there’s less code to review.
- Reduce burnout: Research shows that continuous delivery measurably reduces deployment pain and team burnout. Developers experience less frustration and strain when working with CI/-CD processes. This directly leads to happier and healthier employees and less burnout.
- Recover faster: CI/CD makes it easier to fix issues and recover from incidents (MTTR). Continuous deployment practices mean frequent small software updates so when bugs appear, it’s easier to pin them down. Developers have the option of fixing bugs quickly or rolling back the change so that the customer can get back to work quickly.
