Private Cloud with Open Stack, Docker
-
Upload
davinder-kohli -
Category
Technology
-
view
123 -
download
3
Transcript of Private Cloud with Open Stack, Docker
Private Cloudwith Open Stack, Docker
Increasing organization’s agility and scalability
Davinder KohliJon Reber
Agenda
• Why build private cloud?
• Approach & considerations
• Private cloud ecosystem
• Why Open Stack, Docker?
– Benefits
– Architecture
– Elasticity - Scaling
– Demo
Why Private Cloud?
• NPI data and processing of sensitive information
• Regulatory compliances
• Specific location of data
Data and application security
• Dynamic workload management
• Self-service by developer
Moving from virtualized environment to cloud
• Disaster Recovery
• Provider of niche services
Managed service provider
• Specific infrastructure e.g. HPC - CPU speed, storage, number of concurrent machines
• Competitor to public cloud provider
Support business needs
Infrastructure/OperationsCapacity planningEfficient resource utilizationScalability - Cloud BurstingMonitor utilization
PlatformHigh availability (SLA)PerformanceSupport
Dev TeamEnable agile development & continuous delivery
Infrastructure available as needed
Platform as a serviceVM Templates
Virtual Desktop Solutions
Stakeholders
CxOCapitalize on investments
Existing infrastructure, data centers
Reduce operational costs
GrowthEntry into new marketsNew revenue streamsAccelerate speed to market
Security TeamReduce security threats
Regulatory compliances
Secure app and data
Private Cloud
Approach on adopting private clouds
Cloud Readiness
• Financial
• Business
• Technical
• Organization
Cloud-enabled Platform
• Target architecture
• Capacity Planning
• Delivery processes
• Roles/Responsibilities
Implementation
• Infrastructure setup
• App migration
• Integrations
Considerations – Private Cloud
Use-case driven
Security
Elasticity
Networking
Platform-as-a-service
Metering
Disaster Recovery
•Compute focused - HPC
•Storage focused – data analytics
•Network focused – CDN
•Multi-level
•Scalability - Cloud bursting
•Storage
•Separation of network segments for operators & tenants
•Internal network for Message Bus, Database
•Auto deployments
•VM Templates
•VM/Container management
•Chargebacks
•Managed service provider
•In-house
• Open Source TechnologiesA
• Runs on commodity hardwareB
• Highly scalableC
• Integrates well with infrastructure vendors D
Why Open Stack, Docker?
Open Stack• Public or Private cloud• Apache 2.0 licensed• Releases – Kilo(2015.4), Juno(2014.10), Icehouse (2014.1), Havana (2013.2), Grizzly (2013.1)• Technologies – Python, Rabbit MQ, MySQL/PostgreSQL, libvirt driver, Linux based services,
Django• NOTE: Modular architecture, substituted with other open source technologies based on business
needs
Component Functionality Notes
Keystone Authentication Authenticates users and provides security token
Nova Compute Worker daemon that creates and terminates VMs through hypervisors
Swift Object Storage Persistent; Manage large datasets, VM Images; Chatty; 10x TB; Ref. S3, Multi-tenant, highly scalable
Cinder Block Storage Persistent; Volume storage attached to VMs(/var/lib/nova); 1 TB disk
Glance Image Service Provides images, stored in Swift. Metadata is stored in MySQL DB.
Horizon Dashboard CRUD – instances, network, storage, images
Heat Orchestration Template (HOT) for creating OS resources. Used for auto-scaling, high availability
Ceilometer Metering Runs on each compute node and controller node, polls resource utilization
Neutron Networking Plugs and unplugs ports, creates networks or subnets, and provides IP addressing.
Open Stack - Example Architecture
Open Stack
64-bit Machine
Open Stack - Havana
Ubuntu
glance-api
keystone
nova-scheduler
MySQL
nova -Network
VM
VM
VM
Cloud Controller
Compute Node Compute Node
64-bit Machine
Open Stack
Ubuntu/KVM
VM
VM
VM
Rabbit MQ
Heat
Horizon
ceilometer
Cinder-api, scheduler
Network
64-bit Machine
Ubuntu/KVM
LVM/iSCSI
Cinder -Volume
Block Storage Node
nova -compute
Internet
nova -Network
nova -compute
Ephemeral Storage Node
NFS
Use Case based Architecture
An education company has their online student registration twice a year. Online education content is accessed 75% of the time throughout the academic year. General purpose architecture
Weather simulation software – constant crunching of data and establishing weather patterns.
Compute oriented architecture
Content Delivery Network - streaming video, photographs or any other cloud based repository of data that is distributed to a large number of end users. Network oriented architecture
Once a year, customer load exceeds the capacity of a private cloud of a retail organization, the extra load is handled by a public cloud provider.
Hybrid oriented architecture
Compute Oriented Architecture
Source: Open Stack Architecture Design Manual
Open Stack – Architecture Levers
Compute
• (Overcommit ratio x cores)/virtual cores per instance
• Usage pattern
Storage
• Ephemeral/Persistent
• Content distribution
• Databases
Network
• Speed:1, 10, 40 GbE
• Port count
• Port density
Software
• Operating System & Hypervisor
• Open Stack Components
• Supplemental Components
Form Factor
Business Transaction Volume, Data Retention, Replication, Compliances, Reliability, Usage Pattern
Cost, Data Center Capacity, Performance, Fault Tolerance, Scalability, High Availability
Private Cloud Ecosystem
Physical
Compute NodeLogging Security
PaaS Monitoring
Cloud Controller
Compute Node
Compute Node
Physical Physical
Node Provisioning
Foreman
Puppet
Splunk
Ceilometer
Nagios
LDAP
Oauth
Keystone
Cloud Foundry
Open Stack Auto-Scaling
o User instantiates Heat Templateo Heat registers with Ceilometer for callbacks on alarm eventso Ceilometer triggers alarm event and Heat scales a group based on scaling
policy
o Using monitors on virtual machines or load balancerso Invoke Openstack via REST API or Python cliento Use Chef/Puppet/Ansible
Heat / Ceilometer – Native OpenStack
Other Tools
Auto Scaling with Heat
Controller Compute Node
HOT (scaling.yaml)
ceilometer-api
heat
nova
ceilometer-agent-compute
nova-compute
VM VM VM VM
HOT - Heat Orchestration Template
ceilometer-collector
ceilometer-alarm notification
Containers
VM VM
VMs virtualize the hardware
Limit on the # of VMs based on hardware resources
Maintain multiple OS versions – each guest OS patched/updated separately
A full virtualized system takes mins to start
VM
Physical
Hypervisor
Ubuntu Ubuntu Ubuntu
Host OS
App
Bins/Lib
App
Bins/Lib
App
Bins/Lib
Virtualize an OS – each app running in each containers believes it has full, unshared access to host OS.
Lightweight – runs on single OS and consumes very few system resources (CPU, memory).
Low management – patches applied to single guest OS
Takes seconds to start, no hypervisor overhead
Can run on cheap hardware
Solaris zones, Open VZ, Linux Containers, Imctfy
Container
Physical
Docker Engine
Host OS
App
Bins/Lib
Container
App
Bins/Lib
Container
App
Bins/Lib
Ubuntu Ubuntu Ubuntu
Open source engine that enables (pack, ship, run) any application and its dependencies to be encapsulated as a lightweight container that will run in almost any Linux environment:
Bare metal
Virtualized
Public or private cloud
Docker
Why should I care?
• Faster application delivery• Developers have a platform to run their applications, Ops can deploy
the same container on production servers
• Supports continuous delivery
• Deploy and scale more easily• Runs on dev machine, physical/vm in data center, cloud
• Scale up/down apps/services near real time
• Run more workloads• Place 4 to 6 times as many apps on a single server compared to
virtualization
Docker Concepts
• Written in Go
• Linux kernel features– Namespace layers
– Control Groups – limit resource (memory, CPU, disk I/O) usage
– Advanced multi-layered unification file system (AuFS)
– Container format
Docker Architecture
• Docker daemon – builds, runs, and distributes Dockercontainers on host machine
• Docker Image – build component
• Docker Registry – distribution component
• Docker Container – run component
• Docker client – interacts with daemon using sockets or REST
• Docker Hub – SaaS for managing Docker containers
Containers – Example
Ubuntu 14.4
Ubuntu 14.4
App ContainerWAR
Ubuntu 14.1LB
Ubuntu 14.4
App ContainerWAR
FROM tomcat:8EXPOSE 8080ADD sample.war $CATALINA_HOME/webapps/CMD $CATALINA_HOME/bin/catalina.sh run && tail -f $CATALINA_HOME/logs/catalina.out
Dockerfile (Tomcat):
#!/bin/bashsudo docker build -t javaapp1 .
build.sh (Build image):
#!/bin/bashsudo docker run -it --rm -p 8888:8080 javaapp1
start.sh (Run container):
Containers – Example
Ubuntu 14.4
Ubuntu 14.4
App ContainerWAR
Ubuntu 14.1LB
Ubuntu 14.4
App ContainerWAR
FROM httpd:2.4COPY httpd.conf /usr/local/apache2/conf/httpd.confEXPOSE 80CMD ["httpd-foreground"]
Dockerfile (Load Balancer):
#!/bin/bashsudo docker build -t httpd101 .
build.sh (Build image):
#!/bin/bashsudo docker run -it --rm -p 80:80 httpd101
start.sh (Run container):
QAIntegration
Container based architecture
Dev m/c
Build Server
Private DockerHub
App ContainerWAR
App ContainerWAR
Code Repository
Svc ContainerREST
Svc ContainerSOAP
Continuous Integration
Images
LB
LB
LB
LB
Dockerfile
Source Code
WAR Chef
Scaling Docker with Open Stack
Source: https://wiki.openstack.org/wiki/Docker
Docker Scaling
Docker Machine – Zero to Docker
Swarm - native clustering for Docker containers
Compose – Define multi-container applications running on swarm
2 cores CPU
Open Stack
VM – Swarm Node
Ubuntu
Container Container Container
Docker
RackSpace
VM- Swarm NodeContainer Container Container
Docker
SwarmScheduler
Discovery Service
Physical- Swarm NodeContainer Container Container
Docker
Client
2 cores CPU
What’s are we working on next?
Docker
Baremetal
Chargebacks with
Docker
Open Stack
Contact Information
Davinder KohliHead of Cloud Services
Jon ReberSr. Architect