OpenShift supports Persistent Storage
AUTOMATED CONFIGURATION
SINGLE CONTROL PANEL
CHOICE OF PERSISTENT STORAGE
Ceph RBD
Amazon EBS
Azure Disk
GCE Disk
iSCSI
NFS
GlusterFS
GlusterFS Support in OpenShift
CONTAINER NODE 1
CONTAINER NODE 2
CONTAINER NODE 3
CONTAINER NODE 4
NGINX Container
NGINX Container
NGINX Container
NGINX Container
Scalable Highly-Available Automated Elastic
● Red Hat Gluster Storage runs inside OpenShift in a container (in kubernetes pods)
● Application and storage containers can be co-located
● There’s exactly one RHGS container per host
● Uses host networking, not overlay
● Use OpenShift templates to deploy
● Use a dynamic volume allocator (heketi) to automate volume creation
Launch Container-Native Storage July 2016
VOLUMEA namespace
presented as a POSIX mount point and is
comprised of bricks.
BRICK
The basic unit of storage, represented by an export directory on a
server
SERVER/NODES
Contain the bricks
GLUSTER STORAGE CONCEPTS
GAJAH ANNUAL REPORT 2015 |
BRICKS
/export1
/export2
/export3
/export4
/export5
/export6
/export7
/export8
/export9
/export10
/export11
/export12
/export13
/export14
/export15
STORAGE NODE STORAGE NODE STORAGE NODE
A Brick is the combination of a node and file system (hostname:/dir)
Each brick inherits limits of underlying file system (XFS)
Red Hat Gluster Storage operates at the brick level, not the node level
Ideally, each brick in a volume should be the same size
GAJAH ANNUAL REPORT 2015 |
ELASTIC HASH ALGORITHM
#
BRICK BRICK
0 1 1 0 1 0 1 0
No Central Metadata Server• Suitable for unstructured data storage• No single point of failure
Elastic Hashing• Files assigned to virtual volumes• Virtual volumes assigned to multiple bricks• Volumes easily reassigned on-the-flyLocation Hashed on Filename• No performance bottleneck• Eliminates risk scenarios
GAJAH ANNUAL REPORT 2015 |
VOLUMES
Bricks taken from multiple hostsbecome one addressable unit
High availability as needed
Load balanced data
Managed by Gluster
CLIENTS
SERVER SERVER
BRICK BRICK BRICK BRICK
VOLUMES
● CNS: providing dynamic persistent storage for openshift with GlusterFS in a hyper-converged fashion
● Heketi: the high-level service interface to gluster to manage the lifecycle of volumes in multiple Gluster clusters.
Openshift ⇐⇒ Heketi ⇐⇒ Gluster
What is Container native Storage … and what is Heketi?
● openshift/kubernetes○ dynamic glusterfs provisioner○ glusterfs plugin
● Heketi○ high-level service interface for gluster volume lifecycle
management● glusterfs:
○ one or more glusterfs clusters○ running hyper-converged in openshift
● cns-deploy:○ tool to deploy gluster and heketi into an existing openshift
cluster
Components of CNS
● pod: group of one or more containers that form an entity● persistent volume (PV): to be mounted by application pod● provisioner: to provide PVs upon request● plugin: mechanism to mount the PV, referenced in PV● persistent volume claim (PVC): mechanism for a user to
request a PV● Access types for volumes:
○ RWO - read write once (single node)○ RWX - read write many (multiple nodes)○ ROX - read only many (multiple nodes)
Persistent Storage in OpenShift
● gluster running as pods● aggregating host's disk devices into volumes● gluster storage nodes tied to openshift nodes (disk devices...)● gluster nodes on some, not necessarily on all openshift worker nodes● application pods can be on the openshift nodes (need extra memory!)
gluster cluster in openshift
● containerized systemd○ (running multiple processes: glusterd, brick servers, ...)
● privileged container (systemd, access host /dev, ...)● startup script● host /dev bind-mounted● bind-mount config
○ /etc/glusterfs, /var/lib/glsuterd, /var/log/glusterfs● use host network:
○ gluster ips need to be constant○ gluster config tied to the node○ performance
about the gluster containers
● high-level service interface for managing the lifecycle of gluster volumes● RESTful API and cli ("heketi-cli")● manages one or several gluster clusters● can create, expand, delete volumes (more coming)● hides nitty gritty details of volume creation from caller● just takes size and desired durability type
○ (currently only replicate is supported in CNS)● automatically finds cluster and disks to satisfy the request● stores its state in a database (currently Bolt)● https://github.com/heketi/heketi
about heketi
Top Related