Technology Primer: Monitor Microservices, Docker Containers, Cloud Foundry Containers and More
Journey to the Modern App with Containers, Microservices and Big Data
-
Upload
lightbend -
Category
Technology
-
view
45 -
download
0
Transcript of Journey to the Modern App with Containers, Microservices and Big Data
© 2017 Mesosphere, Inc. All Rights Reserved.
Eryn Muetzel, Product Marketing for Fast Data@errbear3
JOURNEY TO THE MODERN APP WITH CONTAINERS, MICROSERVICES AND BIG DATA
Datacenter Operating System (DC/OS)The simplest way to build, run, scale and manage modern enterprise apps
© 2017 Mesosphere, Inc. All Rights Reserved.
© 2017 Mesosphere, Inc. All Rights Reserved.
INDUSTRY TRANSITIONS
Customer analytics2000s
Electronic customer records1980s
Online customer engagement1990s
Real-time & predictive customer engagement2013+
© 2017 Mesosphere, Inc. All Rights Reserved.
© 2017 Mesosphere, Inc. All Rights Reserved.
WE LIVE IN THE ALWAYS CONNECTED ECONOMY.
3.7BMobile Broadband Users
80%New Data is Unstructured
13Connected Devices per Household
(N. Amer)
40%Annual Enterprise Data Growth
IN THE ALWAYS CONNECTED ECONOMY SOFTWARE IS YOUR BUSINESS.
Build more apps.Personal and meaningful customer experiences
Innovate faster.Capture value from new value streams and actionable data insights
Modernize ITAlways-available services that are secure and cost effective
1M trips daily across 311 cities, disrupting multiple industries.
Personal behavior-based insurance using IoT.
Real-time, personalized clinical care.
MODERN ENTERPRISE APPS IN ACTION.
© 2017 Mesosphere, Inc. All Rights Reserved.
TRADITIONAL ENTERPRISE APPLICATIONS
MODERN ENTERPRISE APPLICATIONS
Data
Code
Latency
Users
2+ Billion smartphone users
3+ Billion internet users
Enterprise data growth (CAGR): 40%+
Source: KPCB Internet Trends 2017, EMC Digital Universe 2014
© 2017 Mesosphere, Inc. All Rights Reserved.
ARCHITECTING FOR SCALE AT TWITTERBEFORE AFTER
© 2017 Mesosphere, Inc. All Rights Reserved.
Streaming
11
THE MODERN ENTERPRISE APPPLATFORM SERVICESMICROSERVICES
STATELESS CONTAINERS ANALYTICS
Batch Machine Learning
Search
DATABASES
Time Series
SQL / NoSQL
MODERN ENTERPRISE APPS REQUIRE NEW CAPABILITIES.
Distributed computing expertise.
Engineering and operations of secure & highly reliable services at scale
DevOps process & culture.
Frequent & reliable releases supported by an automated CI/CD toolchain
Cloud-native technologies.
Containers, microservices, data servies (e.g., Spark, Kafka, Cassandra)
© 2017 Mesosphere, Inc. All Rights Reserved.
MODERN ENTERPRISE APPS
Containers Data Services
MicroservicesOpen source & Social coding1
2
3
4
© 2017 Mesosphere, Inc. All Rights Reserved.
OPEN SOURCE SOFTWARE PROVIDES LEVERAGE
1
© 2017 Mesosphere, Inc. All Rights Reserved.
CONTAINERS SIMPLIFY PACKAGING AND DEPLOYMENT
Private Copy
Shared
App 3
Libraries
Guest Operating
System
Infrastructure
App 1
Libs
Host Operating System
Start / stop time One minute to few seconds Milliseconds
Workload density 10 - 100x1x
App 2
Libraries
Guest Operating
System
App 1
Libraries
Guest Operating
SystemApp 2
Libs
App 3
Libs
Virtual Machines Containers
Host Operating System
Hypervisor
Infrastructure
App 4
Libs
App 5
Libs
2
© 2017 Mesosphere, Inc. All Rights Reserved.
JUST HOW FAST ARE CONTAINERS?
Larry Rau from @Verizon with @flo Launching 50,000 containers in seconds with @mesosphere #DC/OS
© 2017 Mesosphere, Inc. All Rights Reserved.
MICROSERVICES ENABLE FAST AND FREQUENT RELEASES
Traditional Architecture
Many functions in a single process
Siloed teams
REST
API
s
Microservices Architecture
Cross-functional teams organized around capabilities
Scales individually
Each element of functionality defined as “microservices”
Scales monolithically
3
© 2017 Mesosphere, Inc. All Rights Reserved.
DATA SERVICES PROVIDE CONNECTION AND PERSISTENCE
Data Ingestion
Response
Devices
ClientSensors
Event Bus
Reactive App
Database
Analytics
Use Case Examples
● Anomaly detection
● Personalization
● IoT Applications
● Predictive Analytics
● Machine Learning
4
© 2017 Mesosphere, Inc. All Rights Reserved. 19
BUT WHAT ABOUT THE INFRASTRUCTURE?
CaaSPaaS
Container AppContainer App
Stateful Service #1
V1
Big Data Analytics #1
V1
Stateful Service #2
V1
Stateful Service #2
V2
PaaS
Container App
Team A
Big Data Analytics #1
V2
Big Data Analytics
#2V1
Team B
© 2017 Mesosphere, Inc. All Rights Reserved. 20
BUT WHAT ABOUT THE INFRASTRUCTURE?
CaaSPaaS
Container AppContainer App
Stateful Service #1
V1
Big Data Analytics #1
V1
Stateful Service #2
V1
Stateful Service #2
V2
PaaS
Container App
Team A
Big Data Analytics #1
V2
Big Data Analytics
#2V1
Team B
● Extremely low utilization - less than 10%, due to static partitioning
● Manual operations - Weeks to provision, wasted time managing individual machines
● Difficult to experiment with new tech - months to provision environments to evaluate new technologies
● High risk - Failure from manual operations and difficulties maintaining high availability
© 2015 Mesosphere, Inc. All Rights Reserved.
WHERE HAVE WE SEEN THIS PROBLEM BEFORE?
You are launching Google Chrome.Which core would you like to use?
Core 1 Core 2
Core 3 Core 4
You are launching a distributed app requiring 500 cores.Which of your 50,000 cores would you like to use?
Core 1 Core 2 Core 3 Core 4 Core 5
Core 6 Core 8 Core 9 Core 10Core 7
Core 11 Core 13 Core 14 Core 15Core 12
Core 16 Core 18 Core 19 Core 20Core 17
Core 21 Core 23 Core 24 Core 25Core 22
Core 26 Core 28 Core 29 Core 30Core 27
Core 31 Core 33 Core 34 Core 35Core 32
Core 36 Core 38 Core 39 Core 40Core 37
Core 41 Core 43 Core 44 Core 45Core 42
Core 46 Core 48 Core 49 Core 50Core 47
© 2015 Mesosphere, Inc. All Rights Reserved.
MODERN APPS REQUIRE OPERATIONS AT THE DC LEVEL
Traditional App Modern Enterprise App
© 2015 Mesosphere, Inc. All Rights Reserved.
YES, THIS TOO CAN BE SOLVED WITH ABSTRACTION
Scheduler
Executor
Task Launch
Task Launch
Task Status
Task Status
Resource Offer
ExecutorExecutorExecutor
Scheduler
Executor
Task Launch
ExecutorExecutor
Task Status
Scheduler
Executor
Task Launch
ExecutorExecutor
Task Status
Distributed System A Distributed System B Datacenter Operating System Approach
Distributed Systems A+B+C+...
Apache Mesos Two-level Scheduling
© 2017 Mesosphere, Inc. All Rights Reserved. 24
MODERN APPS WITH THE DC/OS MODEL Traditional Approach
Big Data Analytics
Stateful Service
DC/OS Model
Container App
Container App
CaaS PaaS
● Turnkey install of datacenter-wide services● Simplified operations & efficiency● Lowers barriers to experiment with new tech (e.g., Spark)
Container App
Datacenter-scale operating system
© 2017 Mesosphere, Inc. All Rights Reserved.
BRINGING MODERN APPS TO EVERYONE ● 100% Open source
● No limits
26THE DC/OS COMMUNITY
YOUR DATACENTER OR CLOUD IS NOW ONE GIANT
COMPUTER
© 2017 Mesosphere, Inc. All Rights Reserved.
DC/OS SERVICE INSTALL
© 2017 Mesosphere, Inc. All Rights Reserved.
DC/OS SERVICE INSTALL
© 2017 Mesosphere, Inc. All Rights Reserved. 30
DC/OS SERVICE OPS
PLATFORM SERVICES INSTALLED ACROSS DATACENTER OR CLOUD IN A SINGLE COMMAND
© 2017 Mesosphere, Inc. All Rights Reserved.
TRY DC/OS IN UNDER 20 MINUTES
dcos.io
BUSINESS APPS
MESOSPHERE ENTERPRISE DC/OS
CONTAINERORCHESTRATION CI/CD BIG DATA
ANALYTICSMESSAGE
QUEUEDISTRIBUTED
DATABASE SEARCH
PLATFORM SERVICES
RESOURCEMANAGEMENT OPERATIONS SECURITY MULTI-TENANCY
DATACENTER-CLOUD OPERATING SYSTEM
APACHE MESOS
PRIVATE CLOUD AWS INFRASTRUCTURE (EC2) GOOGLE CLOUD MICROSOFT AZURE
MESOSPHERE
ENTERPRISE DC/OS
Container Orchestration
Stateful Data Services
Analytics
Hybrid Cloud
Platform Services Ecosystem
Production Proven at Scale
DATACENTER
MESOSPHERE DC/OS CUSTOMER IMPACTAccelerate Time-to-Market.Capture value from data insights and quickly roll out new services to stay competitive.
Ensure Resilience and Automation.Provide always available and secure business services on hybrid clouds.
Save Money.
Get the most out of infrastructure with high utilization and cost predictability.
© 2017 Mesosphere, Inc. All Rights Reserved.
“We’re running real time analytics and visualization of data from connected sensors
Before DC/OS we had no where near the scale we had today, from thousands of to millions of events per second.
Next, we’re doing predictive GIS”
Adam Mollenkopf, Real Time Geospatial Information Systems Capability Lead
© 2017 Mesosphere, Inc. All Rights Reserved.
“We are building a PaaS and services on an existing OpenStack cloud to enable developers to build microservices on Docker containers and use big data technologies...
DC/OS was a force multiplier for us to get these new services out the door fast...
We transitioned the full enterprise infrastructure at 2X the speed of previous efforts.”
Tim PletcherEngineering Director, Application Services
© 2017 Mesosphere, Inc. All Rights Reserved.
“We came across Mesosphere while planning for the coming IoT market..
If I build everything in a silo, I have no chance...
With DC/OS, I have one single contiguous cluster and I can bring my data and store it and I can run all my applications as well..
and I think that's a huge advantage”
Larry Rau, Director Architecture & Infrastructure, Verizon Labs
© 2017 Mesosphere, Inc. All Rights Reserved.McKinsey Quarterly 2015: “The perils of ignoring software development”Based on assessment of software development projects at 1,300 companies
Top companies bring new software to market
3x faster
© 2017 Mesosphere, Inc. All Rights Reserved.
Thank YouQuestions?
@errbear3 Learn more at mesosphere.com, including our whitepaper:
Modern Enterprise App Operations with DC/OSLessons from Running Containers, Microservices, and Stateful Big Data Services in Production