DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering...

33
® DEVOPS, MICROSERVICES AND THE CLOUD ARCHITECTING AND DELIVERING THE DIGITAL ENTERPRISE

Transcript of DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering...

Page 1: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

DEVOPS, MICROSERVICES AND THE CLOUD

ARCHITECTING AND DELIVERING THE DIGITAL ENTERPRISE

Page 2: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Reinforcing Architecture Disciplines Accelerate Digital Transformation

2

Cloud

SecDevOps

Microservices

Page 3: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®3

Page 4: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®4

Page 5: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®5

Page 6: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®6

Page 7: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®7

Page 8: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

DevOps Unifies Workflows

8

Best Practice Focus• Process execution

simplicity drives faster time to market

• Remove process waste and maximize flow

Page 9: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Rally Around DevOps Workstreams

9

Release Automation

Pipeline Automation

Application Environment ProvisioningSolution

Blueprints

Process

Shared Security Services

Continuous Monitoring (Run-time Insights)

Frameworks

Artifacts

plan develop build test provision validate promote release deploy operate monitor

Continuous Operations

Page 10: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

ImpactsTactical, Strategic, Cultural

Page 11: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Prioritize DevOps Value Targets Using Critical to ROI Analysis

11

26

92

58

73

48

85

60

53

76

46

5350

12

42

2633

22

38

27 24

34

21 24 23

0

10

20

30

40

50

60

70

80

90

100

Continuous

Integration

(build and test

automation)

Solution

Blueprints and

Templated

Infrastructure

Patterns

DevOps Self-

Service

Dashboards

and Playbooks

Automated

Environment

Provisioning

Realistic Lower

Environments

Continuous

Delivery and

Release

Automation

Release and

Rollout

Strategies

Automated

Solution

Validation

Platform

Services

Resilient Auto-

scaling and

healing

Run-time

Telemetry

events

Continuous

Feedback

Dashboards

DevOps: Critical to ROI

Influencing Factor Strength ROI Creation Index

Page 12: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®12

Continuous Integration Continuous DeliveryAgile Planning

Automated Solution ValidationInfrastructure as Code Continuous

DeploymentContinuous Incremental

ReleasesProvision Prod-

Like SolutionContinuous

TestContinuous

Build

Pipeline Phases

Vocabulary

Level 4 Level 5 Level 6 Level 7 Level 8-10

ContinuousAutomation

ComponentsSolution Blueprints

Run-time Insights

Release Automation

Automated Operations

Environment Provisioning

Configuration Management

Package and Test Automation

Developer, Security, and Operations Self Service

Inherited Controls Security Test Strategy

Externalized Security Services

Security Audit Automation

Identity and Access Management Run-Time

ControlsVetted Security Libraries ICAM Deploy

ControlsSecurityComponents

Runtime Telemetry Events and Solution Events

Unit TestReviews Resiliency TestIntegration Test A/B TestVulnerability Test Penetration Test Canary TestAutomated Validation and

Process Triggers

DeploymentSecurityRoadmap Requirements Validation Environment Configuration Identity AnalyticsService LevelsTelemetryTracingApp Logs Security EventsSynthetic MonitoringHealth ChecksNegative Test

Continuous Feedback Loop

Dashboards

Distributed Version Source Code Management Policies

Feature Branch

Release Tag Traffic ShapingImmutable

ImagesPromotion Decision Usage TuningPackage

Infrastructure as Code Artifacts

Auto-ScaleSolution Artifacts

Declarative Definitions

Execute Rollout

Strategy

Soft Launch

Blue/Green

Dark launch

StagedRecreate

RollExecute Release Strategy

Production Shadow

Environment Configuration

Phase Activity

plan develop build test provision validate promote release deploy operate monitor

Continuous Monitoring

Continuous Operations

Automated Resiiancy

Continuous Orechestration

Fail-Safe Deployment

Incremental Release Unit

Component Pattern Repository Blueprints or Libraries Dashboard

DevOps Maturity Roadmap

Legend

Auto-Heal

Page 13: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

0.02.04.06.08.0

10.0Availability

Reliability

Performance

SecurityFlexibility

Maintainability

Time to Market

Assess Your DevOps Maturity Against Value Targets

13

Client Starting Baseline Client Goal

0.02.04.06.08.0

10.0Availability

Reliability

Performance

SecurityFlexibility

Maintainability

Time to Market

Scale Definition Disposition

9 to 10Innovative or ground-breaking application of industry models positively influences the development of new technology applications for the industry

Best in Class

7 to 8

Enterprise, industry, or IT models and best practices drive architecture and design. Scalability, security, extensibility, quality, and performance design principles are documented, implemented, and enforced.

Matches or Exceeds Standards

4 to 6Meets Professional Standard or Standard of Care. Appropriate design standards, methods and tools are consistently used.

Meets Minimum Acceptable Standards

1 to 3 Improperly implemented or neglected but is salvageable with time and effort. Concrete remediation planning has begun. Emerging

0 Absent, broken, unmanageable or improperly architected. Remediation plan is not documented or not funded Absent or Unsound

Page 14: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

DevOps Target Time to Maturity and Value

Startup• 1-3 Month Alpha MVP• 2-6 Month Production MVP

14

Enterprise• 6 Month Alpha MVP• 12 Month Production

Page 15: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

DevOps Cultural Shock

• Holistic Team

• New Attitude

• New Language

• New Focus

15

Page 16: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Success Plan Recommendations

• Build a Tiger Team• Don’t think a FAD diet will succeed • Avoid DevOps Blockers (i.e. test, security, manual governance)• Focus on unblock cards

• Policy• Provisioning• Automation

16

Page 17: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Microservice Level SetPrinciples, Design Patterns, Frameworks

Page 18: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

Lighthouse Project - Digital Transformation In PracticeIdentify Domain Boundaries and

Refactoring Opportunities

Develop Domain Model Create Microservice

Migrate Data or Synchronize

Microservice Data with Legacy System

of Record

Consumer API Gap Analysis

Connect New Consumer(s) or Refactor existing

monolith

Migrate Additional Legacy Consumers

Decommission Legacy Capability

Page 19: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

Target Monolithic Decomposition Objectives with the Appropriate Pattern and Plan for Blockers

Encapsulate and Erode

Encapsulate and Extend

Internal Refactor API Refresh

Consumer Migration Plan

Downstream Synchronization

Channel Consumer

Parity

Disentangling Cross-Domain

References

Which decomposition pattern will initially drive your microservice Proof of Technology (POT) and Proof of Concept (POC) Lighthouse project?

Page 20: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

Gauge Architecture, Technology, Process, and Team Strengths within key Microservice Strategy Areas

Reshape Architecture

Domain Driven Design and

API ContractsMicroservice

PatternsTechnical Reference

Architecture

Identity Context and

Least Privilege

Refresh Technology

Microservice Framework

Container Packaging and Orchestration

Runtime Insights

(monitor, trace, alert, manage)

Eventing and Caching

Reframe Collaboration

Contract First Development

API Intake Process, API Gap Analysis

and API Portal

DevOps and Site Reliability Engineering

Inline Decisions based on Big Data analytics

and ML

Reinvent Opportunity

API Economy and Channels

API Composition (Experience,

Process, Domain,

Connector)

Product Engineering

Real-time notification and

capability domains

Page 21: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

Prioritize POT Roadmap Items within each Microservice Initiative Workstream

Domain Driven Design

Business Domain Model

Data flows and Work

flows

Consumer Interaction Patterns

API Contracts

Microservice Framework

Wire Protocol and

Serialization Framework

Core Listener, Client, and

Trace Frameworks

Configuration and Secrets

Health Endpoint,

Metrics, and Logging

Eventingand

Caching Framework

Cloud hosting and

DevOpsContainer Platform

Networking and API gateway

CI/CD Pipeline

Runtime Insights

Security and Identity

Access and Authorization

Tokens

Trust Relationships

and Gateways

Identity Provider

Integration

Container Scanning and

Least Privilege

API Adoption API Portal API Gap

AnalysisAPI

Coaching

Page 22: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Point me to Your Business Domain Model

Product Catalogue Recommendations Shopping

Cart

Supply ChainManagement

CustomerManagement Payment

OrderIdentity ManagementFulfillment Identity Management

Page 23: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Domain Decomposition Example: Identity

UserSelf-Service

UserManagement

UserSingle Sign-

On

UserPermissions

ProtocolsSingle Sign-OnSingle Sign-Out

Create usersInvite users

Force password resetActivate / Inactivate users

User self-registerEmail confirmationForgot password

Change username / emailChange password

Enable MFA

Manage user permissions

User Profile

Manage user profile

UserPermissions

User Profile

Manage tenant permissionsManage application policies

Accounts, MembershipCRM Integration

Users

Config

Users UsersPolicies

UserPermissions

TenantPermissions

User Profile

Accounts

Customers

Identity Sub-Domain Policy Sub-Domain Accounts Sub-Domain

Page 24: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Unwinding the system of record problem

Config API Users API

UsersConfig

User Management

Web

ConfigWeb

User Self-Service

Web

IdentityServerWeb

Config Runtime

API

Config Management

API

User Runtime

API

User Management

API

User Self-Service

API

Users Users UserSelf-Svc

UserRuntime

Page 25: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

API Host Configuration

Enforce Domain Design with API Contract First Design and Code Gen

Code Generation

{…}{…}{…}

API Client Libraries

API Controllers

Domain Models / Events

https://openapi-generator.tech/

Page 26: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Contract PR - Sanity Check Aggregates

User

User Logins

UserStatus

User Claims

User Aggregate

Permissions Aggregate

Profile AggregatePermission

Profile

ContactInfo

userid

userid

Page 27: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Cloud ServicesPaaS First

Page 28: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Cloud Hosted Infrastructure Foundation

• Cloud is a DevOps forcing function• Dynamic provisioning and scaling• Monitoring integration with operations management (continuous

orchestration – topology, traffic)• Cloud PaaS accelerates microservices

• API Gateways and Service Mesh• Service Clusters (i.e. Kubernetes)• Eventing• Identity and Security• Runtime Insights

28

Page 29: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Achieve Consistency ViaEvent Source Pattern

Resource AAPI

ReadStore A

Projector A

Event Stream

ReadStore B

Projector B

Event Sourcing

System of Record

Resource ARead API

Page 30: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Logs should be meaningful (and Structured)

Not informativeBut, something to act on

Informative

Wut?

Page 31: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Service Dashboards

Runtime Infrastructure

DevOps Processes

Telemetry

Operations Management

Alerts

Alert Analysis

Structured Logs

Feed

s

Events

NFR KPI / SLA

Issue/Ticket Generation

Business KPI / SLA

Site Reliability Engineering Console

Trend Definition

Root Cause AnalysisDependency MappingStatus Reporting

Anomaly Detection

Management Automation

Monitors

Probes

Trend Analysis

Runtime Insight Data

Structured Logs

Runtime Utilization

Runtime Performance

Runtime Topology API Events

Topology

Impact Analysis Root Cause Analysis (RCA)

Insight Engine

Runtime Insight At Scale

Actions

Page 32: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy

®

Fast, Continuous Feedback Loop

Recap: Target Improvements Within Every Workflow Phase

Create Plan for Feature

Release & Deploy

Validate

Demote Feature

Build Test

Provision

Monitor PromoteiSign

Approval

Develop

Operate

Legend:

Failure Feedback

Decision

Process Go Feedback

Release with Confidence

Automated Testing

Streamline and Automate processes

Rapid Delivery CycleIn Days - Weeks

Continuous Monitoring (Run-

time Insights)

Continuous, Automated Inspection of Each Stage

Reduce and Ultimately Eliminate the Accreditation GapApplication

Environment Provisioning

Continuous Operations

Pipeline Automation

Solution Blueprints

Dashboard Shows Readiness at a Glance

32

Release Automation

Page 33: DEVOPS, MICROSERVICES AND THE CLOUD · 2019-11-13 · DevOps and Site Reliability Engineering Inline Decisions based on Big Data analytics and ML Reinvent Opportunity API Economy