Docker Chennai Meetup - OpenEBS Overview
Transcript of Docker Chennai Meetup - OpenEBS Overview
![Page 1: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/1.jpg)
Storage as microservice (a.k.a OpenEBS)
Built using Docker
Docker Chennai Meetup
@openebs
#containerizedstorageforcontainers
3 June 2017
![Page 2: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/2.jpg)
Let us revisit containers
Image source : https://newsroom.netapp.com/blogs/containers-vs-vms/
VM
Container
![Page 3: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/3.jpg)
Docker -> Portability
![Page 4: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/4.jpg)
Docker registry → DevOps bridging
![Page 5: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/5.jpg)
Storage controllers software version control
Monolithic Storage Kernel
iSCSI Software
A
ReplicationSnapshot
EncryptionB
QoS&
MgmtC
volume1
volume2
Storage software is same for all
volumes
![Page 6: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/6.jpg)
Should we containerize storage ?
Version control of application software is made easy
through Docker images
How about the storage software ?
![Page 7: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/7.jpg)
When storage is containerized
Docker Hub
![Page 8: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/8.jpg)
Storage can also be a micro service
Containerized Storage Kernel
(OpenEBS)
volume1
Storage ContainerVSM 1
SoftwareA, B, C
volume2
Storage ContainerVSM 2
SoftwareA’, B’, C
![Page 9: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/9.jpg)
Dockerized storage
https://hub.docker.com/r/openebs/jiva/
![Page 10: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/10.jpg)
What is OpenEBS ?
➔ OpenEBS is containerized storage for containers
➔ OpenEBS solves the persistent storage issue for containers
Legacy
storage
X
Application
Pod
★ Application pod management
automatically takes care of storage
volume management
![Page 11: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/11.jpg)
Dedicated mode of deployment
Separate hosts
![Page 12: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/12.jpg)
Network
Data
OpenEBS Maya
Hyperconverged mode of deployment
K8s master
K8s minions
VSM or Storage PodApplication Pod
TCMU/iSCSI
OpenEBS Maya Storage
Orchestrator
Application ( yaml )
![Page 13: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/13.jpg)
What is Maya ?
Maya
![Page 14: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/14.jpg)
Manifests express intent
K8S used with special care for apps requiring
persistence because they require brittle tight coupling.
Container 1 Container 2 Container 2
Hard wired connections via plug-in
stateless stateful
NAS SAN S3 NAS SAN S3
Without OpenEBS DevOps workflow is broken
![Page 15: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/15.jpg)
Manifests express intent
stateless stateful
No changes to DevOps workflow even for containers requiring
persistence. Users manifest their intent and the storage and
storage controllers adjust automatically as needed.
OpenEBSPolicy engine
Remote Storage
Local Storage
OpenEBS Storage Hosts
OpenEBS VSMs
OpenEBS Storage ClusterEBS
APIs
Containers and underlying storage,
local on host or dedicated storage
pods OR remote S3 or EBS storage
all grouped into a storage cloud that
just works. Billing by AWS.
Manifests express intent
With OpenEBS the workflow works
![Page 16: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/16.jpg)
DevOps operations with OpenEBS
OpenEBS got the storage ops off my back!
Because of OpenEBS we are rolling into
production.
![Page 17: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/17.jpg)
How is OpenEBS built ?Storage containerization
Storage orchestration
Deep storage operations
Core Storage features
is built using
is integrated with
is built into (OpenEBS Maya)
is built into (OpenEBS)
Smart Capacity Management Volume Latency Monitoring and Auto-healingAdapt to varying workloads - Application Aware Storage (Zones)AWS EBS Like APIsStorage Analytics!
Enterprise HA Intelligent QoSDistributed Scaleout Smart Tiering for higher performance
![Page 18: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/18.jpg)
OpenEBS - Word of mouth
Containerized storage for containers
Calico / Flannel owns networking for K8S. OpenEBS owns
storage.
The only open source solution built for this. CEPH is not built
to be containerized.
I run (use) OpenEBS just like I run (use) K8S (EBS).
OpenEBS got the storage ops off my back! Because of OpenEBS
we are rolling into production.
![Page 19: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/19.jpg)
Persistent storage is still an issue today
https://clusterhq.com/2016/07/17/dockercon-ecosystem-clusterhq/
"Containers have moved
beyond ephemeral storage
to persistent storage to
hold data and protect your
applications."- 451
Research
Mesosphere Declares
‘Container 2.0,’ the Stateful
Era
Kubernetes Version 1.3
include additional
resources for supporting
stateful containers
![Page 20: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/20.jpg)
Persistence Storage Types
https://portworx.com/stateful-enterprise-applications-docker/
Flocker
Convoy
Rex-ray
Torus
Infinit
PortWorx
OpenEBS*
Docker Plugin
K8s Plugin
![Page 21: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/21.jpg)
Typical Use cases
![Page 22: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/22.jpg)
Where does OpenEBS fit?
DevOps and containers are the futureWhich is already here, but unevenly distributed
Real computing needs persistenceBut - persistence for DevOps & containers is broken
![Page 23: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/23.jpg)
Usecase : Reuse existing SAN in
your cloud to build container friendly storage
platform
![Page 24: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/24.jpg)
Build container friendly storage platform
AWS
VMs
EBSDocker
Application Pod (Application has
container-native storage)
● The existing SAN storage (EBS or GCP PD or
any other block storage) in your cloud can be
integrated into K8s application pods by
OpenEBS
● Persistent storage need not be managed
separately in your cloud
● Your persistent storage is managed from within
your application yaml files
![Page 25: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/25.jpg)
Usecase : Build AWS EBS or GCP PD type platform On-Premise
![Page 26: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/26.jpg)
Deploy EBS style platform On-Premise (Dedicated)
Your data center
Bare metal servers
Docker engines (Compute)
Bare metal storage
servers
Docker engines (OpenEBS)
Application
pod
yaml
A single application yaml
file controls both the
compute and storage
needs
![Page 27: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/27.jpg)
Deploy EBS style platform On-Premise (Hyperconverged)
Your data center
Bare metal servers
Docker engines (Compute+OpenEBS)
Application
pod
yaml
A single application yaml
file controls both the
compute and storage
needs
![Page 28: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/28.jpg)
Comparing AWS EBS & OpenEBS
Feature AWS EBS OpenEBS
Network Block Storage as a
service
Yes No. This is a technology
that can be used to build a
service or platform like AWS
EBS
Technology form factor Unknown Open Source
High Availability grade Five nines of availability Five nines of availability
Availability zones Yes Yes
Point in time Snapshots to
S3
Yes Yes, to S3 or any S3
equivalent (locally hosted
Minio for example)
![Page 29: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/29.jpg)
Connectivity in AWS EBS (Optimized for Cloud service)
Login into AWS Console
Create an EBS volume
Attach to an instance
Format and put it to use
by application
![Page 30: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/30.jpg)
Connectivity in OpenEBS (Optimized for Container service)
Login into Kubernetes
Set up an application
yaml with OpenEBS as
the storage provider
Volume is created on
OpenEBS and mounted
on K8S minion
Format and put it to use
by application
kubectl create -f demo-mysql-iscsi.yaml
![Page 31: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/31.jpg)
Snapshots on AWS EBS vs OpenEBS
Incremental
Snapshot
Restore
Amazon
Cloud
Enterprise
data center
Incremental
Snapshot
Restore
Amazon
CloudAny other S3 Cloud
storage provider
S3 storage
in the data center
Scality
Minio
OpenEBS
volumes
Apps in
Containers
Incremental
Snapshot
Restore
Incremental
Snapshot
Restore
Incremental
Snapshot
Restore
![Page 32: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/32.jpg)
Other features comparison of AWS EBS & OpenEBS
Feature AWS EBS OpenEBS
Volume migration across tiers
(Magnetic, General Purpose
SSD, Provisioned IOPS)
Yes Yes, VSMs can be migrated
seamlessly across tiers.
QoS control Yes (Provisioned IOPS) Yes, granular control of IOPS
on a per volume basis
Burst IOPS, credit bucket for
unused IOPS
Yes Yes. Min/Max IOPS, burst
IOPS, credit bucket
functionality
Auto scaling of the storage
nodes
Yes (Not sure. Internal to EBS
architecture)
Yes. OpenEBS storage nodes
are added to clusters to allow
auto scaling of either capacity
or performance
![Page 33: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/33.jpg)
OpenEBS Roadmap
0.1
0.4
1.0
0.3
0.2
● Soft launch / Basic version
● Containerized controller
● Longhorn integration basics
0.2 - Basic k8s integration
0.3 - Full k8s integration(Hyper-Converged)
0.4 - Tiering and distributed storage
1.0 - First enterprise edition(Full functionality for basic production usage)
K8s Provisioning via EBS-like driver
S3 building blocks
Complete longhorn integration
First usable release for community
consumption
Complete K8s integration
Fully hyper-converged
Support for local backing store
Building blocks of Q0S
Building blocks of Tiering
Tiering and QoS demonstrated
Building blocks of ML for storage analytics
DevOps operator use cases demonstrated
Tiering and QoS demonstrated
Building blocks of ML for storage analytics
DevOps operator use cases demonstrated
Nov, 2017
Sep, 2017
Jun, 2017
Mar, 2017
Jan, 2017
![Page 34: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/34.jpg)
Take a peek at our github project and star us if you like
https://github.com/login/oauth/autho
rize?client_id=768d7617e686c9c2b
d81&scope=user%20public_repo&s
tate=openebs1234cloudbyte1234&
allow_signup=true
github.com/openebs
![Page 35: Docker Chennai Meetup - OpenEBS Overview](https://reader031.fdocuments.us/reader031/viewer/2022030318/5a64b3e37f8b9a88148b479f/html5/thumbnails/35.jpg)
What do you think?
https://github.com/openebs
Join the community
#slackSlack.openebs.io
blog.openebs.io
@openebs