VMware vSAN Vision - nlvmug.com · VMware vSAN Vision Duncan Epping - @DuncanYB Chief Technologist...
Transcript of VMware vSAN Vision - nlvmug.com · VMware vSAN Vision Duncan Epping - @DuncanYB Chief Technologist...
1
VMware vSAN Vision
Duncan Epping - @DuncanYB
Chief Technologist – Storage & Availability
Data: tremendous business value potential…
Information Growth50 x data growth
from 2010 to 2020
Source: IDC* and EMC *April 2014
Where is all that data going again?
The rise of software-defined storage
Why?
Today’s storage products do not meet the requirements of the
evolving IT industry
LUNs SUCK
Storage Devices Becoming Faster with Better Endurance
$5 $20$1
IOPS
250K
500K
1M
750K
$/Gig
$15$10
Tomorrow
SSD is the new Capacity Disk
High capacity NVMe
Byte-addressable NVDIMMs
Network latency >> Device Latency
Today
Low latency devices relatively
expensive for persistent storage
Device latency >> Network latency
NVDIMM
DRAM DRAM
CONFIDENTIAL
This is where we are today…
Hyper-Converged powered by VMware vSAN and vSphere
Runs on any standard x86 server
Pools HDD/SSD into a shared datastore
Delivers enterprise-grade scale and performance
Managed through per-VM storage policies
Deeply integrated with the VMware stack
vSphere vSAN
vSAN Datastore
VMware customers trust their infrastructure to vSAN
vSAN Customers
Countries Deployed
Critical Apps on All-Flash
>7,000 >100 83%
Leading HCI Vendor#1
Supporting a broad variety of use cases
vSAN
Business Critical Apps Virtual Desktops (VDI)
DR / DA
Test/DevDMZ
ROBOManagement
Staging
Really Simple Setup
Storage policies
VM / VMDK policy profile
Policy Gold
Availability
Capacity
reservation
IOPS Limits
Value
FTT = 2
Space efficient
40GB
1000
Moving to application lifecycle management through policy
Policy
Placement and configuration by
policy
Control of QoS at VM / VMDK
level
Simple, scalable automation
platform
Primary Cluster
vSAN
Virtual Volumes
VAIO
vSphere
vSAN objects and components
Component-2
Mirror Copy
Component-1
witness
Mirror Copy
RAID-1
vSAN is an Object Store
Each Object has multiple Components
This to allow you to meet Availability and
Performance requirements
Data is distributed based on VM Storage
Policy
Fault Domains, increasing availability through rack awareness
• Create fault domains to increase availability
• 8 node cluster with 4 defined fault domains (2 nodes in each)
FD1 = esxi-01, esxi-02 FD3 = esxi-05, esxi-06
FD2 = esxi-03, esxi-04 FD4 = esxi-7, esxi-08
• To protect against one rack failure only 2 replicas are required and a witness across 3 failure domains!
15
FD2 FD3 FD4
esxi-01
esxi-02
esxi-03
esxi-04
esxi-05
esxi-06
esxi-07
esxi-08
FD1
vmdk vmdk witness
RAID-1
vSAN – Stretched ClusterActive-Active data centers
• vSAN cluster split across 2 sites!
• Each site is a Fault Domain (FD)
• Site-level protection with zero data loss and near-instantaneous recovery
• Support for up to 5ms RTT latency between data sites
– 10Gbps bandwidth expectation
• Witness VM can reside anywhere
– 200ms RTT latency
– 100Mbps bandwidth required at most
• Automated failover
5ms RTT, 10GbE
Today
VMware vSphere & vSAN
vSphere
witness
vmdk vmdk
witness
Space efficiency
Nearline deduplication and compression per disk group level
– Enabled on a cluster level
– Deduplicated when de-staging from cache tier to capacity tier
– Fixed block length deduplication (4KB Blocks)
– Compression after deduplication
RAID-5 and RAID-6 (Inline Erasure coding)
– RAID-5 needs a 3+1 configuration, but only 33% overhead
– RAID-6 needs a 4+2 configuration, but only a 50% overhead
Provide block storage through vSAN iSCSI
vSAN iSCSI Target Service enables Block Storage!
– Support for Oracle RAC
– Storage for physical workloads
– Provides max ~1K LUNS per Cluster
Provides all core vSAN functionality for the iSCSI target
– Dedupe and Compression, RAID-1, RAID-5, RAID-6
iSCSI Target
iSCSI Initiator iSCSI Initiator
iSCSI Network
iSCSI Object
iSCSI Object
vSAN
Datastore
2-node Direct Connect and Witness traffic separation
vSAN Datastore
witness
10GbE vsan traffic
management & witness traffic
Ability to connect the two nodes directly using Ethernet cables
Separating the vSAN data traffic from witness traffic
Two cables between hosts for higher availability of network
Allows for Layer-2 and Layer-3 topologies and strict separation
of traffic streams
1GbE Switch
vSAN 5.5March 2014
vSAN 6.0March 2015
All Flash Configuration
64 node vSAN cluster
Health UI
Rack Awareness
vSAN 6.2March 2016
Dedupe and Compression
RAID 5/6 support
Software Checksum
IOPS Limits
Performance Service
Enhanced Capacity Views
vSAN 6.1September 2015
Stretched Cluster
Replication - 5 Minutes RPO
2-node ROBO
Health Monitoring & Remediation
vSAN 6.5November 2016
iSCSI Services
2-Node Direct Connect
This is where we are going…
Three key pillars for the future
Availability and Security
Management and Monitoring
Data Management and Mobility
Decrease operational effort through pro-active
health monitoring and analytics.
Increase availability through advanced fault domain
architectures and encryption of data at rest
Policy based data management from
on-premises to public cloud.
INFRASTRUCTURE MANAGEMENT
Site failure or outage
Cluster-level recovery
Planned migration
Monitoring
vSphere vSAN
From Infrastructure Management in the Private Cloud…
Infrastructure Management
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
…To Data Management In the Hybrid Cloud
vSphere vSAN
Data Repair and Recovery
File or DB Recovery
Data Archival
Data Analytics
Data Governance
Data Management
Test and Development
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Stretched Cluster: Local Failure Protection
Overview
• Provide local host and disk group failure protection
• In case of site failures there will still be local availability within the surviving site
• RAID1 of RAID5/RAID6/RAID1 is supported on All-Flash
• RAID1 of RAID1 is supported on Hybrid
Benefits
• Host and disk group protection when site failure happens
• Local replication when component has failed
vSphere vSAN vSphere vSAN
Primary Cluster
vSphere vSAN
Primary Cluster
vSphere vSAN
5ms RTT, 10GbE
RAID-1
RAID-5 RAID-5
Witness Location
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Increase flexibility with Site Affinity for Stretched Clusters
• User can specify single site location of VM’s components if site level protection is unnecessary
• Policy driven using SPBM
• Reduces network and storage requirements
• Ideal for solutions that already use application redundancy (Exchange DAGs, SQL Availability groups, etc.)
vSphere vSAN vSphere vSAN
Primary Cluster
vSphere vSAN
Primary Cluster
vSphere vSAN
RAID-0
RAID-6
Witness Location
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
vSAN Encryption
Overview
• Software Encryption for Data at Rest
• Enabled on a per cluster level
• Integration with major central key management technologies that are KMIP compliant
• Works with de-duplication, compression
• Order: 1. Checksum, 2. de-dup, 3. compress, 4. Encryption
• Cache Tier is also encrypted!
Benefits
• Enable customers who require security provided by data at rest encryption for deploying vSAN
• While still benefitting from advanced data services
• Protection against security threats with unauthorized accessing/hacking of sensitive data on drives
• Safely transferring drives to other facilities
vSphere vSAN
vSAN Datastore
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Infrastructure Management in vSAN Covers Day 0-2 and DC Wide Operations
Single-click upgrade for
ESXi, vCenter Server and
vSAN including firmware,
drivers, and software
Day 0
Network usage,
re-sync/rebuild traffic,
query disk groups for
read/write latencies, etc.
PowerCLI, S.M.A.R.T
data for hardware
monitoring, and health
checks on ESXi when
vCenter Server is down
Day 2
Capacity trends
for dedupe, improved
topology view for
Stretched Cluster,
and unified alerts
Integration through SPBM
DC-wide
Auto-ConfigCluster Health
PerformanceService
vRealizeOperations
vRealizeAutomation
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Intelligent, automated operations with vSAN Easy Install
• Provides easy way to bootstrap new
vCenter Server on a single host
• vSAN configuration steps built into
VCSA installer workflows
• Simplifies process of building out new
environmentvSphere vSAN
vSAN Datastore
VCSA
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
VUM Integration
VUM
vSphere vSAN
vSAN Datastore
Overview
• Simple upgrade / updates of drivers and firmware!
• Automatic new version check and notification
• Built-in baseline/baseline group creation for vSAN clusters
• Automatic downloading of ESXi image
• Support online mode with internet access and offline mode with local image repository
Benefits
• Simplified upgrade / updates for vSAN clusters
• Simplified driver and firmware management
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Using Analytics for Effortless Infrastructure Management
VMware Support
Customer
VMware Product Team
VMware Cloud
Telemetry Data Collection
Systems Modeling Predictive Algorithm Statistical Analysis
vSphere vSAN
vSphere vSAN
Customer 1
Customer N
On Premises
data
answers answers
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Extending policies to data management
Data policies
All protected files matching search criteria
Policy Gold
Replication
Backup
Domain
Anti-Affinity
Zone
RPO = 10 mins
7 Day Retention
US
EMEA
Data Management policy per App,
VM, VMDK, File or Object
Policies across Private and Public
Clouds
Global monitoring and compliance
reportingSecurity Encryption
NFS DD BOOSTS3 API
Google AWS Data Domain
Secondary Storage
Primary Cluster
vSphere vSAN
NAS
VMDK Object File
ValuePolicy
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Scalable, Efficient Native vSAN Snapshots
Feature Benefit
High
Scalability
• Natively built into vSAN storage level
• Designed to scale up to limits of vSAN
Highly
Performant
• Low impact to overall vSAN I/O even when large
number of snapshots are created
• Performance impact is fixed, regardless of
snapshot depth
Unified
solution
• Foundational technology for a unified approach –
Ad-hoc snapshots, local protection, cross cluster
replication, and archival to external storage
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
vSAN Native Archival Protection
Archivecopies
Localcopies
Feature Benefit
Archival of older snapshots to an
external NFS storage array
• Retain a longer history of
snapshots cost-effectively
Isolated copies of snapshots
available in case of total vSAN
failure
• Rebuild VMs onto a vSAN
using the snapshots stored on
external NAS array
Policy-based configuration of
archival
• Archival frequency and
retention policies are an
extension of vSAN policiesPrimary Cluster
vSphere vSAN Secondary Storage
NAS
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
End goal: Full data mobility
Cloud
Primary Cluster
vSphere vSAN
DR Site
vSphere vSAN
ArchiveSnapshots
LocalSnapshots
LocalSnapshots
DRSnapshots
5 minutes RPO5 minutes RPO1 Day retention
1 year
Object
Space efficient native
snapshots
Flexible scheduling and
retention policies
Replication of snapshots to
secondary storage and DR site
Snapshot based Local
Recovery, DR and Long
Term archival
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
So what’s left?
Future: File system abstraction for CNA
Native Distributed File System
Photon Machine
vSphere vSAN vSphere vSAN
Efficient image distribution
Scalable data volume sharing
Consistent snapshots / clonesmulti-container stateful applications
Primary Cluster
vSphere vSAN
Primary Cluster
vSphere vSAN
Photon Machine Photon Machine Photon MachinePhoton Machine
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Native vSAN NFS/SMB File System on vSAN
Potential Approach• Highly integrated vSAN file services solution
• Supports large number of clients
Management• All management including installation, configuration, share
management via vCenter Server
Protocol Support• NFS
• SMB
• ??
vSphere vSAN
vSAN Datastore
Distributed File System
File system protocol Server
Client Client Client Client Client
This presentation contains product features that are currently under development. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Wrapping up…
VMware vSAN enabling hyper-converged for all workloads!
vSphere vSAN
vSAN Datastore
vmfs block file restObject