A checklist for teams moving to DevOps
Introduction
Since Microsoft understands the need for open-source platforms, they are fueling more open-source developments in recent days. Microsoft rebranded its cloud-based CI/CD platform VSTS into Azure DevOps services in September 2018. They announced free DevOps services for all the open-source projects around the same time. Apart from the open-source model, still there are lots of customers who are using the Microsoft technology stack using Azure technologies and especially Azure DevOps. We see lots of articles and whitepapers talking about DevOps nowadays. They mostly talk about open-source tools and technologies. Through this article, I am planning to throw an insight into the Azure DevOps platform and services. We will discuss how are these practices bringing a change in the way how the industry is operating already. We will have an overview of what tools and technologies Azure DevOps offers for DevOps practices. Most importantly, we will cover the required skills and learning opportunities for those who are moving to Azure DevOps.
Though certain hints are provided about the tools, it is the reader’s responsibility to study whether those tools will fit the particular requirement.
Target audience
- Consultants/Architects who have been part of defining the technology roadmap and strategy for IT industries
- Engineers planning to move to DevOps (not necessarily Azure DevOps)
- IT managers to understand their roles with DevOps practices (not necessarily Azure DevOps)
Waterfall vs Agile vs DevOps
Let’s look at different development methodologies in terms of delivery.
The waterfall model has a lengthy feedback cycle where the requirements of the software are analyzed, designed, developed, unit tested, and released for customer acceptance testing. Since we are talking about the software as a whole, it usually takes months to get feedback for the software, especially from end-users. Often times the delayed feedback leads to lots of re-work in the shipped product. There is less scope for accommodating the changes in customer expectations over time. There are situations where the product been scrapped altogether when major deviations found.
The agile model on the other hand, instead of putting lots of effort into traditional, structured, and bureaucratic approaches, offers a way of flexible development that accommodates changes over time. Usually, deliveries to production will take weeks. In essence, the products are developed in iterations. In each iteration, there is a feedback loop (sprint reviews, etc.,) where the product owners will have a chance to look into how the product is been developed. They can test the developed features, give feedback for improvements. The feedback can be accommodated in the upcoming iterations. By a shorter feedback loop, the quantum of rework is reduced, which offers a window for accommodating changes. Basically agile model offers the following features over the waterfall model:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
DevOps in essence is not a replacement model for Agile. Perhaps it brings the best outcomes of DevOps and reducing the release/feedback cycles shorter. Shortening of the release cycle is achieved by bringing the development and operations team closer and best utilizing various automation tools in the build and release process. Upon code check-in, code is analyzed for defects, build, unit testing, various functional and integration tests are executed, released to UAT environments for customer feedback. If the feature meets all the expectations, it can be released to production. Incremental changes are added in days in production.
DevOps
DevOps is a software development strategy that bridges the gap between the developers and the IT staff. With DevOps, organizations can release small features very quickly and incorporate the feedback which they receive, very quickly. It reduces the probability of software failures and shortens the lead time between failures.
The stages of software development, delivery, and monitoring of production systems are done by teams that are closely coordinating and collaborating. The development team is responsible for the activities depicted left portion of the above figure viz., plan, code, build and test. Whereas the operations team is responsible for the activities depicted on the right side viz., release, deploy, operate and monitor the production systems. The development team is not going to take the responsibilities of the operation team or vice versa; instead, the development team is better at understanding what is expected to make the operations better. Similarly, the operations team understands what is expected to make the development better. So, both of these going side-by-side by supporting each other to strengthen the overall delivery process.
The various automation tools are getting into the picture at this juncture to make both the development and operations side process seamless. By automating all possible steps, the manual dependencies/errors are reduced, delays in-between each step are avoided, the efficiency of the team increased by getting feedback on time.
The transition from Engineering practices into DevOps Engineering
We will discuss few key aspects when you are aiming to transform your existing software engineering practices into DevOps practices.
- Decide the server model
- Choose the source code repository and Application Life cycle Management tool
- Defining the delivery roadmap
- Imparting training to team
- Cultural change
- Implement the delivery chain
- Defining the DoD, “Definition of Done”
- Continuous learning and improvement
Decide the model of the infrastructure
This step is to decide where are your infrastructure is going to reside. There are different options available. When you plan to have all or part of your services intended to run on the cloud, then there are three different cloud options viz., public, private and hybrid. Each one has its own advantages and limitations.
Public cloud offers large-scale standardized infrastructure accessed via public networks. It is suitable for startups and SME’s. It has a scalable infrastructure based on demand.
The private cloud provides computing power as a service within a virtual environment. However, the resources are accessible by a specific organization that owns them. It offers extended security and privacy over the public cloud. It is suitable for large enterprises and public sector organizations.
A hybrid cloud uses a mix of on-premises, private cloud, and public cloud capabilities based on different needs. It gives greater flexibility and more deployment options. This model is again suitable for large enterprises and public sector organizations.
Choose the source code repository
The source code repository is the place where the source code and other important artifacts are stored, versioned, and controlled. For the source code repository, there are various platforms available in the market. There are centralized and distributed options. The distributed model especially Git is nowadays popular with DevOps-based deliveries. Few key factors that you should keep in mind while choosing the source code repository: the tool has greater flexibility for team collaboration such as code reviews, optional deliveries such as cherry-picking, gated check-ins such as pull requests. Local regulations and security policies may decide the model for source control repository whether it should be located on-premises or on the cloud. Cloud-based repositories are popular among DevOps teams.
Choose the branching and merging strategy
Source code branching and merging decide the way how the source code is being managed, various review and gated check-in process, how the continuous integration and continuous deliveries are picked up. Release branch, feature branch, and task branch are the popular options among agile teams. The release branch is to have a branch taken out of the master branch per release. All the changes for the release are applied to the same branch and master branch. Feature branch and task branch on the other hand have a branch for each feature and task/bug respectively. Each branching model has its own advantages and limitations. Task and feature branches are most popular among DevOps teams.
Define the code review strategy
Code review is a key aspect where potential improvements in code are identified in the early stage, make the team learns from mistakes, enhances collaboration among the team. Self and peer reviews are very important in DevOps teams. There may be more rounds of reviews by different stakeholders at a later stage.
Choose the Application Life cycle Management(ALM) tool
Application Life cycle Management(ALM) provides a comprehensive way of managing the requirements, backlogs, project and release planning, and testing. It also offers a way for collaboration and communication among the DevOps team and other stakeholders. There are various open-source and proprietary tools available in the market for ALM. Explore and decide the right ones that fit your need.
Defining the delivery roadmap
Upon the model of infrastructure is decided, the next step is to define the delivery model. As we discussed before, DevOps practices bring the best part of the agile development model with improved collaboration with the operations team. The following figure shows how continuous integration (CI), continuous delivery (CD), agile development, and DevOps are connected.
A typical delivery model with an example with a toolset is depicted in the following figure. Each numbered items are self-explanatory of their intended functions. One important thing to highlight here is on each phase when there are deviations found, there will be options for course correction in a less painful way. For example, if a unit test case failure leaked during coding, the developer will get to know upon code check-in as the building checkpoint validates the unit test cases and report if there are failures.
Another aspect of delivery is to define various stages of release. The following figure depicts a sample flow of control with different stages. The different stages viz., build (development), test, staging, and production environments are having different quality checks.
Build (Development): Upon the code is self and peer-reviewed, pull request analyze the code for design, security, and other coding expectations, validates whether the code is integrating with source code repository with updates from other developers, run the unit tests to test whether the developed unit is doing it’s intended function and check the code coverage whether the expected portion of code is tested by automated tests. If any of these defined checkpoints are not met, then the integration process with getting interrupted and intimated to the development team for correction. Upon all defined criteria in the development server are passed, the build artifacts are produced that can be used for further releases.
Test: When the artifacts are released to the test environment, various functional, load, performance, integration, and security tests are validating the code. There may be more validations at this stage based on industry requirements. In case any of these tests failed, then typically a bug is created in the team’s backlog in the ALM tool. The bug is then prioritized, fixed, passes through all review and integration validations, again re-tested in a test environment.
Staging: Staging is the pre-production environment where final sanity checks are made before releasing to production. Sometimes this is UAT phase where user acceptance tests are being carried out. So, there is a scope for automating certain user acceptance test cases even at this stage.
Production: This is where actual end users are using the features developed. The end user’s observations and feedback from various monitoring tools are important inputs for defining further backlogs from a release perspective.
Tools for CI/CD
With the shorter release time in mind, automation at various stages plays a vital role. There are plenty of tools available to choose from for continuous integration, delivery, test automation, and monitoring purpose. Choose the right toolchain that fits based on the organizational tests.
Define the guidelines for development (and keep it improving with lessons and technology updates)
There are standard best practices for each technology and tool. Oftentimes it is evident that there will be a conflict between different practices. To avoid anomalies and to keep the DevOps team focusing on what is relevant, better define the team’s own guidelines for coding and other practices. In order to cope up with the latest expectations, keep the guidelines revisited on each iteration, update with the lessons and latest technology updates. Continuous improvement is one of the key aspects of best-performing teams.
Imparting training to team
Since the expectations of the conventional release process are changing, it is a good idea to keep the team trained with the tools and processes selected. There are various training courses available online from different training providers. Choose the right course that fits based on your technical stack and tools in DeOps pipeline.
Cultural change
This is another key aspect of efficient DevOps practices. Without bringing cultural change to the team, DevOps maybe just in books. DevOps team should have more autonomy than a command-and-control way of doing things. Let the team volunteer for the tasks instead of assigning them. Keep the business objectives revealed to the team so that the team prioritizes things. Make the changes in the process visible to the team. Encourage the team to raise questions. Bring the trust among the team that all are operating for the unified goal. Collective learning should be motivated instead of criticizing others. Celebrate the learning and success regularly internally. Rethink and redefine the processes if those are not fits. Each member in a DevOps team will act like an IT manager who owns their deliverable, coordinate with peers for team deliverable, coordinate with other stakeholders and operations team, and so on.
Defining the DoD, “Definition of Done”
By defining the definition of done, the team can focus on different aspects of delivery and ensures whether those are met. It also reduces the chances of chaos at a later stage about what is expected as the delivery expectation. DoD may vary from product to product and organization to organization. Setting minimum quality expectations, meeting security guidelines, coding guidelines, review process, expected automated test cases, etc., are a few of the checkpoints in the DoD.
Summary
We had an overview of various aspects associated with setting up DevOps practices.