Agile In DevOps
Several significant concepts are related to DevOps. Agile, Lean, and ITIL are just those concepts. This section is about Agile and its relevance to DevOps. Patrick Dubois and Andrew Clay Shafer attended the Agile 2008 conference in Toronto. At that conference, Andrew presented a Birds-of-a-Feather session on Agile infrastructure. Patrick was in attendance and listened to the presentation. After they talked, Andrew presented on Agile infrastructure at Velocity Conference the following year. In 2009, Patrick started a conference in his hometown of Ghent, Belgium. He called the meeting DevOps Days, effectively creating the term DevOps and beginning the DevOps movement. DevOps has since caught on in a big way with its history rooted in Agile. You might be familiar with Agile. The manifesto for Agile Software Development was written in 2001 by a group of software developers dissatisfied with the current state of software development. They felt the increasing levels of bureaucracy and process were being layered onto projects in the hopes of better results. The outcome was often the opposite. The previous approach to software development is called waterfall. And that’s because it moves software down from stage to stage. First, you get all the requirements wholly done and documented. Then you send the requirements to development, which codes them. Development sends the code to QA, who tests it. QA then sends it over to release engineering.
- Agile In DevOps
- Lean In DevOps
- Lean Philosophy
- ITIL, ITSM, and SDLC
- ITIL Guidelines
- Phoenix Project
In Agile development, the process is purposefully more iterative. Instead of trying to complete each phase upfront, it stresses flexible collaboration between workers and customers around the frequent interim deliverables of working software. This can quickly generate solutions that address customer needs with fewer lingering quality problems. And Agile has proven its benefits. 85% of Agile teams have seen increased productivity and 80% report faster time to market. Critics of Agile think that since it’s quicker and more collaborative, it must be sloppy and random. Agile teams also report better delivery predictability in 81% of cases and enhanced software quality in 79% cases. Learning about Agile is a significant endeavor. The library has many courses that can help you learn more about Agile. However, if you’ve read the principles in the Agile Manifesto, you’ll see what’s missing; any mention of operation. Agile talks about working software, but it wasn’t customary to bring system administrators into the product team. Also, the manifesto doesn’t mention the last part of the software delivery pipeline, where infrastructure’s built and the apps are deployed and maintained in production. In the beginning, Agile was seen as a threat by the infrastructure side of the house in IT organizations. Many IT professionals had to be convinced that it wasn’t crazy by development managers. Most experts that have moved to DevOps report they would never go back. So, is DevOps the same thing as Agile? No, you can practice DevOps without Agile and vice versa, but it can and probably should be implemented as an Agile extension since DevOps has such strong roots in Agile. When I was asked to write a DevOps manifesto, after consideration, I decided that very slight edits of the Agile Manifesto captured the heart of it. Replaced software with systems and added operations to the list of stakeholders. And the result is a solid foundation to guide your DevOps journey. DevOps isn’t just an outgrowth of Agile. It owes a lot to lean software.
Lean In DevOps
Lean is a systematic process for stopping waste and was created in the manufacturing world by W. Edwards Deming and Taiichi Ohno’s Toyota Production System. It revolutionized the Japanese industrial economy after World War II and later returned to the United States. The book, “Lean Software Development An Agile Toolkit,” adapted these lean techniques to software development. They identified seven principles of lean that apply to the software. Like the just-in-time tenet of lean manufacturing and aligned with the agile idea of being flexible, you try to move fast but delay decisions in enhanced feedback loops and group context. Building integrity will inform the approaches to continue this integration and testing.
The basic philosophy of lean is about identifying which actions you and your organization perform that add value to the product or service you produce and which do not. Activities that don’t add value are called waste. Lean recognizes three significant types of waste, and they all have Japanese names: muda, muri, and mura. Muri is the primary form of waste, and it comes in two types: type one, which is technically waste but necessary for some reason, like compliance, and type two, which is just plain wasteful. The Poppendiecks also defined seven primary wastes that are endemic in software development. This includes bugs and delays, but it also includes spending effort on features that aren’t needed. Toyota didn’t take long to adapt the lean to product development. A recent popular adaptation of that is found in Eric Ries’s book “Lean Startup.” In the book, he proposes the build-measure-learn loop as a variation of the usual Kaizen plan-do-check-act cycle. You focus on delivering the minimum viable product to customers, get their feedback, and iterate from there instead of trying to analyze what the perfect product would have been upfront. There are a variety of techniques that go along with lean.
Kaizen is one, and another is value-stream mapping, where you analyze the entire pathway of value creation and understand what exact value is added where, how long it takes, and where waste resides in that pathway. In lean product development, that value stream is referred to as concept to cash, the entire path from the idea to its realization, including all the production and distribution required to get it to customers. DevOps was only indirectly influenced by lean when it started, but it didn’t take long for the ideas and their relevance to what DevOps was trying to accomplish to be discovered. Jez Humble, author of “Continuous Delivery,” proposed modifying the CAMS set of DevOps principles to CALMS to include lean, and John Willis largely agreed with him. “The Goal,” “Lean Software Development,” “Lean Startup,” and “Lean Enterprise” are four good starter books that can teach you the most about lean. Lean has become essential to DevOps, especially in successful DevOps implementations. The Puppet Labs State of DevOps Report identifies lean management as one of the two important practices that contribute most to organizational success.
ITIL, ITSM, and SDLC
The Information Technology Infrastructure Library is a set of detailed practices for IT activities such as IT service management and IT asset management that focus on aligning IT services with the needs of the business. DevOps stands on the shoulders of giants. And there are a lot of concepts from the various ITSM and SDLC frameworks and maturity models that are worth learning. Teams should be organized around the standard ITIL processes, with sections for change management, supplier management, incident management, etc. But when you implement these processes, you want to use a lean and agile mindset and need to craft them in a way that’s people first, and that doesn’t introduce waste or bottlenecks, into the value stream, in the name of a standard or best practices.
IT service management is a realization that service delivery is an integral part of the overall software development life cycle. Engineers should properly manage it from design, development, deployment, and maintenance to retirement. In the past, the software development life cycles focused on code writing and tended to stop at handoff, or if they mentioned deployment and maintenance, they went into very little detail on them. In this way, ITSM is one of DevOps’ ancestors. ITIL was the first ITSM framework. It launched the idea of ITSM. So many folks still speak about them as if they’re the same thing, even though other ITSM frameworks, like COBIT, have emerged since. ITIL is a UK government standard that grew out of the Thatcherism of the 1980s as the previously organically managed IT assets.
The UK government didn’t have a lot of alignment and standards. And so, their central IT division published guidelines on managing services in the late eighties and early nineties. The UK’s central IT group did this first version of ITIL so well that it piqued interest outside the UK government. In 2001, ITIL v2 was published with the explicit intent of being used by others. V3 was published in 2007 and updated in 2011. It uses a process model-based view of controlling and managing services. It can be said to inherit from Deming’s plan due check act cycle. ITIL recognizes four primary phases of the service lifestyle. Service strategy, design, transition, and operation. It has guidance for every kind of IT process you’ve ever heard of, from incident management to portfolio management, to capacity management to serve catalogs. At the same time, all of the high-level principles of ITIL make sense. It’s designed to be a reasonably prescriptive and top-down framework. While it’s not technically against ITIL to do agile development or perform continuous integration and deployment or other such practices, honestly, much of the culture, advice, and consultancy around ITIL assumes a waterfall push-driven model of the world. But it certainly doesn’t have to be that way.
Gene Kim, the author of the “Phoenix Project” and the “DevOps Cookbook” and probably the most avid DevOps evangelist today, founded the IT Process Institute, a nonprofit dedicated to studying IT processes and analyzing best practices around IT performance. In 1999, Kim and Kevin Behr and George Spafford published a slim hundred-page book, “The Visible Ops Handbook, Implementing ITIL in Four Practical and Auditable Steps,” in 2004. It describes not only the core parts of ITIL that correlate to an IT organization’s success but puts forward a roadmap to implement them in a lightweight manner. “Visible Ops” spread slowly as a cult classic IT book originally. The book’s content was great, but the implications of the book were also genuinely electric. The idea is that you could have well-defined control processes that are compliant and auditable. Still, they could be light and agile, opposite most ITIL implementations people had experienced. So, learn from ITIL and its brethren, but think twice before implementing a specific reference process. Instead, consider the process’s goal and analyze how you would accomplish that in a lean DevOps manner. From CI to the chaos monkey, many DevOps are about taking an alternate path to the intended goal and finding out it’s better than the way established IT wisdom might tell you to take.