Executive Guide to DevOps and Deployment Automation

Dave Sayers talks DevOps for the Enterprise and discusses how deployment automation can hold the key to happy troops and business innovation success

Executive Guide to Enterprise DevOps and Deployment Automation

Executive Guide to DevOps and Deployment Automation

1

DevOps in the Enterprise

Business is fast and getting faster.

In fact for many enterprises the speed of getting new products and improved services to market is defining every boardroom metrics from share price to customer retention and churn rates or market share. So building your organisation to be able to react quickly and deploy improvements with speed but without the usual associated risks is becoming more than just a topical movement. This eBook discusses some of the issues and considerations you might want to put on your next senior management agenda and discuss with Line of Business, Development and Operations colleagues. So let’s talk about DevOps and some of the principles of this relatively new movement within IT and then we’ll take a look at how these principles have been applied in enterprise organizations. Dave Sayers guides us through some of the principles and current thinking. Dave, over to you.

People over process over tools

The first aspect of DevOps is around culture and aligning the focus of different parts of an organization to a common goal. The challenge of delivery in today's organization and the DevOps movement is to address the gap between the development side of our organization and our operations. This is a coming together of two organizations. I'll talk to you around how we realized that.


2

2

On the left-hand side of the diagram we've identified the first gap between the line of business and development. This is mostly being dealt with by agile solutions; the tooling and the process, is quite established in this area around Lean and Kanban and all kinds of processes around Scrum for example. The second gap in this kind of new ALM lifecycle is where vendors like us operate, addressing this gap between development and operations. From a people perspective in the first instance, historically development has been about the creation of value for organizations. So development focus is value creation and it's the way of bringing new ideas to market in IT. On the opposite side of the spectrum is operations, where its focus is on value protection - reducing the amount of change that has been introduced into an organization (as traditionally change has been associated with risk - less change therefore by nature appeared to mean less risk). Operations are traditionally incentivised on stability, performance, uptime, etc. This creates a misalignment of incentives, which has different parts of your organization focusing on and rewarded for different types of behaviours. Businesses are being required to deliver change more quickly and a large and ever increasing part of that delivery mechanism is online and through technology. Business is driving these requirements and through necessity technology delivery is having to react. There are lots of agile development tools and tool chains that provide developers with rich capability to develop code quickly and work collaboratively.


3

And then, the cloud capability that’s out there with technologies from the likes of Amazon Web Services public cloud or virtualisation from VMware and others for a private infrastructure or cloud capability, many organisation have reduced infrastructure provisioning from what would not have been uncommon to be eight weeks to provision a physical server down to minutes to spin up a VM. The massive improvements and relatively maturity in agile and cloud capabilities have highlighted the inadequate processes and tools of operations - in the relay race of bringing new technology and products to market, operations teams are not only last to take the baton but are also now the ones that are now in many cases the weakest part of the team.

The bottom of the diagram is key aspect of DevOps that we’re particularly talking about today and what a lot of people are talking about is how do we apply a consistent set of processes, a consistent of set of methodologies, and potentially a consistent set of tools across all of our environments from our first initial environment to our production environment in a consistent, efficient way.


4

4

If you haven't read this book, The Visible Ops by Gene Kim it's definitely worth a read and this is how he defines DevOps as collaborative working relationship between development and IT operations resulting in a high deploy rate in the context of what we're talking about here, while simultaneously increasing reliability, stability, resilience and security. I think that sums it up pretty nicely.

MidVision helps companies deliver

high deploy rates of product and

service improvements and

innovation in even the most complex

technology environments while

ensuring operational integrity,

security and resilience by using

deployment automation.


5

Okay. So let's just look at some of the principles of DevOps.

Collaboration

DevOps is a collaboration across disciplines. At a high level those are two groups are really development and operations, but there are sub-components within there. We have testing, and performance monitoring, infrastructure provisioning and operations management, and it's about all of those teams working together collectively for a common goal.

Develop and test in the same environment as production

From a technology perspective, we're looking to develop and test our code and configuration and the infrastructure against production-like systems straight from the start. Development runtime might not be the same scale as production - but in principle other than scale they should be the same. We might not be talking about 64 nodes or anything like that for developers to test on, but we are saying that the platforms look the same; the operating systems, versions, configurations and the code, so that we know we are testing against the platforms that we're going to be using ultimately in production.

Deploy in small increments

Another key component of DevOps is around deploying in small increments. In fact the whole Agile development methodology is really around iteratively developing components and releasing them quickly. DevOps in many ways mirrors this aspect of agile delivery, rather than having this waterfall infrastructure management approach where we have 30 people in the office over a weekend where we do a big bang release - we are moving towards smaller change more frequently. Applying a high level automation underpins this - and without it the net effect would be seen to increase risk - as done manually more change will like result in more outages.


6

6

The implementation of advanced DevOps strategies, processes, tools, etc. has really come from Big Web. For the most part natural selection has resulted in those who innovate the fastest as the winners - and reliably releasing updates in small batches are a hallmark of many of these organisations. In the enterprise we're working with organizations to apply these similar principles. So, for example, we are regularly deploying - making daily or weekly deployments - to Internet banking systems for large corporates and that's a real move away from where they were not too long ago when they were still applying waterfall management techniques for their infrastructure management. Many thought leaders at the forefront of DevOps theory may not see daily or weekly application updates as revolutionary - but whilst enterprise organisations often don’t operate at the scale or big web applications are often a lot more complex and the environment that they work under have very different influences.

Continually validate

Then another key aspect is that we need to continually validate the operational and quality characteristics of our environment. If we're going to start introducing change regularly, we need to understand the behaviour it is having on our environment.

Moving away from Silos – ‘done’ means the code is released

DevOps means moving away from the silos approach of developers throwing code over the fence to infrastructure people, and saying my job is finished. There's a move in this collaborative work environment, for "done" meaning the code is released/deployed into our production system. That's when the development team is finished. Not when they've compiled it. Developers move further through the journey and collaborate with the infrastructure and operations people until we're actually satisfied with the product on the shelf and being used by consumers.


7

Version controlling everything

We should be version-controlling everything. Traditionally, most organizations are Source Controlling the code that they compile – e.g. the Java or Perl or whatever it is that they’re writing in - and then build, version it and then release that. DevOps is really about version controlling everything. This is all of our configuration definitions for our targets, database config., web servers definitions, all the runtime configuration of our app servers, application configuration, etc. . Everything is put into Source Control - and we're releasing things in the same way that we release our code but to our IT operations environment. We've touched on this frequent quick release concept now and aside from the fact we are able to frequently release components, another two aspects of releasing frequently.

Continuous delivery

Continuous deliver is an extension of continuous integration. Rather than just building and unit testing, with continuous delivery we are then deploying into our target systems and hopefully also running automated tests against them. An advanced implementation of this is to then have automated quality gates to progress on a pipeline towards production i.e. from and development onto QA for example. Automated quality gates are not something I have seen in any enterprise organisation – that is not to say they do not exist, but the norm would be to put manual approval processes in place (at least on the last leg of the "path to production" Another aspect of this is in terms of removing bottlenecks. It’s giving people who need the ability to perform actions like deployments and promotions, the ability to do it themselves. So, self service again is something that is very much promoted - and aids the continual flow of change towards production by increasing efficiency of the release process.


8

8

Don’t just test the code, test the release process

What we are also looking to do is not just test the code but also the automation and infrastructure configuration process and we are doing that as we flow through all of those environments. We are not just testing the fact that we've deployed our code, we are also testing the infrastructure configuration and then we’ve got tests for both aspects of that.

What does DevOps mean in operations?

Automate everything that you can

The model a lot of organizations have had traditionally – where people are logging in to visual consoles and changing configurations and then that doesn't work and then change it again – if you're going to implement a DevOps strategy, you need to be automating everything you can. Those configuration definitions as we said should be stored in the Source Control system. Then this gives us that desired state for our run time, so we have the ability then to know what the environment should look like and what it actually does look like.

Instrument pervasively

From an operations perspective, instrumenting data, collecting data and understanding trends is about allowing us to understand the effects of the change in our environments. And then in terms of actually how we manage our run time environments from an operations perspective can change as well. The organizations that are at the forefront of this thinking, that will do things like canary releases where they releases subsets of changes out to particular parts of their user base. If we are going to an Agile iterative process of introducing change quickly, then what organizations are increasingly able to do with technology is to put changes out to 10% and then 25% of the user base and whilst their instrumenting and understanding the effects on their environment (They understand and they're not seeing errors. It worked, okay.) Then that can be rolled out through high levels of automation to the rest of the system. This is another area that is much more prevalent in big web or tech organisations - but this should be an object of enterprise organisations as well.


9

Mean time to recovery (MTTR) vs. Mean Time Between Failure (MTBF)

People sometimes talk about these competing things – about mean times of recovery verses the mean time between failures. So, the point here that I'm trying to make is that it's not always a key identifier to say, we've had a number of failures of these components within our environment – if they're highly available and the system hasn't gone down then this isn't actually causing us an issue and it's the recovery of those aspects that are key. So it’s bringing that capacity back on line so that we are not exposed and a part of that is around re-provisioning, not repairing. So we are recovering to a known state so we can push out a new web server from our desired state to expand the definitions, rather than trying to spend a lot of time fixing one that's there already.

Culturally collaborating - really?

In the enterprise, I would say the cultural aspects of DevOps is in my opinion, not one that's been given a huge amount of focus. We're not seeing these same alignments. Very progressive IT firms talk about in terms of the problem being the enemy and a healthy attitude toward failure. I think that what we're trying to do and what a lot of organizations are doing in the enterprise is really reducing the time between developers producing assets and then being released into production, and collapsing that and making that more efficient. That's how DevOps have been applied rather than this one team where we all work together and the world's great. I don't see these changes being made in the same way that I hear it from the Silicon Valley-styled firms. But you can't really talk about DevOps without mentioning some of these differences.

Putting the science back into computer science

Some of these DevOps principles are really around putting some of science back into computer science. This is agreeing we understand what our IT infrastructure looks like, we have a desired state of that, we can change it and we know the effects of that and we are moving away from this being some kind of art form where "Steve knows about our portal environment and if he's not here then we're in trouble" to actually understanding what our environments actually look like and a scientific approach to managing them.


10

10

The commoditisation of IT

Another aspect of DevOps and at a high level on the journey towards automation, allows us to really commoditize some of the components that we have within our environment. So, if we want a new infrastructure test environment or performance test environment, these become much more commoditized assets. This means that we're able to clone and provision and reproduce assets at will. We are able to expand a state to deal with this capacity. We are able to decommission it when we don't need it and re-provision it when we do need it and this is really just allowing us to commoditize those components and not be afraid to change those pieces. In terms of that commoditization, I think that there are two ways that people have tried to apply this to the enterprise. Probably the most topical one is the Platform as a Service, where IT organizations offer platforms internally to their consumers to build and release applications.

Platform-as-a-Service

So PaaS is a term that seems to be banded around now and everyone has a different interpretation of what it is or means. From how I see it being implement it is a multi component standard stack - so that could be a platform based on Oracle or a WebSphere application server or MySQL, JBoss, and Apache. They'll offer these standard platforms to their consumers with a self service type portal that's driven by HPOO or some kind of interface for consumers to actually pull down a platform, provision their business applications it's on this standard platform, test them and then let them be decommissioned and through the test cycle and then again offering those as live services.

Infrastructure-as-a-Service

Then another sort of configured implementation is Infrastructure as a Service (IaaS) where rather than an entire platform, that are multi-component configured at certain levels, potentially with different versions of internal applications already deployed, we might say from an infrastructure and service perspective, we can pick and choose, fire up virtual machines, with databases configured and those types of things. I'd say this is one of the effects that DevOps is having in that it's allowing us to provision these types of components. I'd say this one of the common ways it's been realized in the enterprise.


11

Delivering change doesn’t mean you have to like your eggs scrambled!

One of the problems many organisations have is a fear around introducing change - and that there's a fear that the release mechanism itself is a cause for concern. An analogy I made to someone talking about this recently, was is if we related this to Just In Time delivery systems within retailers, then this is analogous to the management team saying we can't have our eggs delivered five times a day because those guys keep tripping up and smashing them all. Right? We need to know that when we're trying to deliver a change, that it is going to get there safely. And that's, again, one of the requirements for a high level of automation - the consistency, the understanding that what we deploy really gives us that assurance that the delivery mechanism itself is going to work.

A core principle: We’re testing the release process itself and not just the release.

So if you look at the statistics in this area around production failures, organizations like Gartner and others are suggesting that as much as 70% of IT outages are the results of errors and mis-configuration made by operations teams. So, if that's the case within your organization today, then I guess it's pretty understandable if people are reluctant to increase the amount of change if there's a concern that this is going to result in outages. The process of testing the actual mechanism will give us some assurance that when we're deploying into production three times a day or twice a week, or whatever is - that it's going to work.


12

12

Tooling and tool-chains

At MidVision when we started five years ago we were really looking at the automation of manual tasks - installing software, configuring it, deploying it, maintaining it. Now, what we've done as a vendor is that we have integrated with tools around us to form a tool chain. This splits down into these main areas: The version-controlled library which comes in two forms, really. One is its Source Control system, SCM systems, like Clearcase, Subversion, RTC, GIT, etc. where we are writing all of our configurations to - so we write them out for Source Control. And the other library is a library of assets for packages that have been created. In ITIL terms this is a Definitive Software Library (DSL). These are products like Artifactory and Nexus Sonatype, Rational Asset Manager, etc. So, these are tools that we've integrated to from MidVision. The third area is around the modeling of our system. We've got a model of what our automation pattern should look like. Then we've got policies, dependencies that exist within that model that can be applied across many different environments. We apply the same model of our automation and the activities that we want to carry out on to different environments, therefore testing that process as we go.


13

Now, one of the things I should mention is that if you want to implement a DevOps tool chain, then you are going to need all of these tools, but you can choose not to do that or choose to do it in a stepping stone approach where by pieces of some of the tools are put in and they're built on. A common scenario that we see is organizations that say "Well, we can implement the automation engine (ours is called RapidDeploy) to integrate with Source Control and maybe Jenkins because that's what we've got. Then we'll implement a DSL or other things later if we choose to." Or they may choose to the automation engine even without integrating with those other tools. And that's all possible and you still get some of the benefits, but if you want to implement a DevOps tool-chain end to end that gives you audit and roll back and compliance around who made what changes and when, then I think the most efficient with the most capabilities requires you to have a tall tool chain that looks similar to the chart above.

Have you got further questions about Enterprise DevOps and Deployment Automation?

Do you still have unanswered questions about automated deployment and how it would work in your organisation? Please get in touch with MidVision and we’ll be more than happy to answer questions for you.

http://info.midvision.com/request-a-free-30-day-rapiddeploy-trial

Executive Guide to DevOps and Deployment Automation

Business

Transcript of Executive Guide to DevOps and Deployment Automation