DevOps

As a consultant, I tend to work with a variety of clients and teams all across the product maturity spectrum.

Some are just starting; maybe they have an MVP, maybe they are still building it. Others have existed in their space for years. Typically, when I get called into projects, the product maturity is on one extreme of the spectrum. DevOps maturity, on the other hand, tends to follow a different distribution with most DevOps programs somewhere past just starting, but not quite mature.

 

In this not-quite-mature stage of DevOps, there is usually a sort of CI/CD pipeline, automatic tests, linters and an emphasis on automation. Logs, metrics, and backups are taken automatically and centralized. There may be basic dashboards and saved queries. Documentation may or may not be adequate for helping to debug problems. If there was a DevOps checklist, all the boxes would be checked. At this point, many teams stop building.

 

Perhaps there was never a plan to mature beyond that level. People love to say “premature optimization is the root of all evil”, and they have a point; if one of my clients didn’t have automated backups the first thing I’d do is enable them, at a minimum by taking snapshots on the VM level. If they didn’t have any logging setup at all the first step would be to enable them, maybe push them to centralized storage. Once a team reaches that level of maturity they have a working system. Backups don’t usually need to be restored that often, and a team can get by looking at the logs there only when something needs to be debugged. When the team isn’t getting woken up in the middle of the night there is less incentive to continue maturing.

 

Another reason why DevOps engineers may shy away from working to mature their program is that after a certain level it becomes “less fun”. The rate of building slows down and the focus shifts to maintenance and metrics. Metrics are decidedly Not Cool; they’re what non-tech people and management types use to justify their paycheck. Still, once the product reaches a certain level and developers stop making large amounts of process change, defining metrics and direction will help to continuously improve the program and keep it from getting out of date.

 

In a mature DevOps program, much more time is spent optimizing than implementing. Logs have long been centralized and now most of the effort is spent looking for new indicators that something may be wrong. Linters may have more custom checks than out of the box code formatters and style checks. Mature DevOps is about continuously building processes and responding to change by continuously optimizing and improving. 

 

Mature DevOps programs allow errors to be caught earlier and make them easier to debug. In an early stage DevOps program, an administrator may only get alerted if the entire application is down, but may not notice if a small component is broken. For example, as the program matures, the team may start to track the rough number of “404 page not found” responses returned, alerting when the metric crosses a certain threshold. Later, the team can get even more granular, perhaps reporting when a 404 is returned for a page that had previously not returned one.

 

Mature DevOps helps reduce tech debt and improves efficiency. A basic DevOps program may only track the overall page load time or memory usage. As a result, by the time the many small statistically insignificant changes become noticeable, it has already become infeasible to go back and identify/fix them. If code changes are reviewed on their performance impact to individual components, then adverse changes can be identified earlier in the process, increasing the overall health of the application and extending the amount of time before the inevitable “major refactor”.

 

If DevOps is your thing, and you want the opportunity to build out advanced DevOps programs for different types of clients, RunAsCloud is hiring. Reach out to careers@runascloud.com to learn more! 

As we all know in this era of technology, manual tasks are gradually becoming obsolete.

Everyone in the industry expects that processes, phases of development and deployments are expedited. The achievement of CI/CD (Continuous Integration and Continuous Deployment) plays an astounding role in automating every phase of the software development lifecycle.

 

The basic phases for CI/CD are CodeCommit, Build, Test and Deploy. With the help of a CI/CD pipeline, the process of delivering the product to a customer has become faster and less risky. Here I will discuss various tools involved in each phase of the CI/CD pipeline.

 

Let’s consider a scenario to explain CI/CD in a better way. Imagine that you are going to build a web application that is going to be deployed on live web servers. For this project, you need developers who will write code to build this application. Each day after writing code for this application, it has to be stored somewhere. There are various tools for committing code like git, and svn and when it tracks history it is known as a version control system.

 

Introduction to tools used in phases of CI/CD Pipeline

We use various Continuous integration tools like Jenkins, Bamboo and many others that are available in the market. As Jenkins is an open-source tool, it is broadly used and has a large number of built-in plugins that are used to integrate with other tools. A few important ones are Git plugins, Maven, Ant, Docker, etc.

 

CI tools check for any new CodeCommits in a given interval of time we can be preset or can be set in such a way that once any code is committed, it will pull it from the version control tool and download into the workspace for further process. It then automatically builds and tests every change that is committed. We can define the entire job or the task in Jenkins.

 

Next, it goes through the build phase where tools like Maven, Gradle, etc can be used. In this phase, the code is tested, compiled and packaged in war/jar format by the build tool. J-unit generates the test report.

 

Once the build phase is over, then you move on to the testing phase. In this phase, various additional testing tools like selenium and sonarqube can be involved and integrated with a CI tool like Jenkins for code quality test, code coverage, integration testing, and user acceptance testing.

 

Next, we can move to the deploy phase in the pipeline which includes deploying the application in a staging server or production server as per the company needs. There are various ways to deploy, like deploying the application in Docker Container, Simple deployment to a webserver EC2, deploy using AWS elastic Beanstalk, or AWS EC2 Container Service (ECS) for cluster-based applications. There are many more ways to do this. Even tools like chef, puppet, or ansible can also be used as a deployment tool.

 

Once the application is deployed and live in the production server we can go ahead and do continuous monitoring. There are many monitoring tools like Nagios, Splunk, AWS Cloudwatch, etc. which can be integrated with the CI tool. Some important reasons to use a monitoring tool are that it resolves any system errors (low memory, unreachable server, etc.) before they have any negative impact on your business productivity. It can be used to automatically fix problems when they are detected and can also send an alert to the specific team or individual for any issue that occurred.

 

Additionally, the pipeline can be configured so that if there are any errors in any phase, the CI tool can define the task or pipeline as a code. This code can then be emailed back to the development team or testing team, alerting them to which phase has failed so that they can fix it. Then they will push it into the version control system and goes back into the pipeline.

The diagram below shows a summary of CI/CD pipeline: