Enterprise-gradeHybrid Hyper-Scale Microsoft Cloud OS Open.
-
Upload
rosanna-lloyd -
Category
Documents
-
view
223 -
download
3
Transcript of Enterprise-gradeHybrid Hyper-Scale Microsoft Cloud OS Open.
Spark the future.
May 4 – 8, 2015Chicago, IL
Deploying and Managing Hyper Scale Applications on Azure VMsAlan Stephenson – Azure Program ManagerGuy Bowerman – Azure Program Manager
BRK3733
Cloud Compute continuum – IaaS -> PaaS
Building blocks - Units of work Deploying resources in Azure Resource
Manager at scale Managing apps with higher level
services
What’s in this session?
Enterprise-grade Hybrid Hyper-Scale
Customer
ServiceProviderMicrosoft
ConsistentPlatform
ONE
Microsoft Cloud OS
Open
Cloud Compute Continuum
Ultimate Control
Rapid Development
VMs /VM Scale Sets
VM ExtensionsService Fabric /Batch
Websites / App Platform
IaaS PaaS
VMs and VM Scale Sets
Azure Public CloudAzure Pack Private Cloud
VM Extensions
SCALR, RightScale, Mesos, Swarm
Service Fabric (VMs and
Containers)Batch
App Service Media
Web Apps
MobileApps
Apprenda, CloudFoundryOpenShift, Jelastic, Marathon
Infrastructure
IaaS and IaaS+
General Compute
PaaS
Vertical Compute
PaaS
VMs and VM Scale Sets
Azure Public CloudAzure Stack Private Cloud
VM Extensions
SCALR, RightScale, Mesos, Swarm
Service Fabric (VMs and
Containers)Batch
App Service Media
Web Apps
MobileApps
Apprenda, CloudFoundryOpenShift, Jelastic, Marathon
Infrastructure
IaaS and IaaS+
General Compute
PaaS
Vertical Compute
PaaS
Open Choice at Every Layer
Cluster Orchestratio
n
HDInsight
VMs and VM Scale Sets
Azure Public Cloud
VM Extensions
SCALR, RightScale, Mesos, Swarm
Batch
Media
HDInsight
IoT Finance Media Oil / Gas …Most
industries
Some Scale Application Scenarios
Visual Assets
VM
Frame Description
VM
VM
Rendered Frame
Rendered Frame
Rendered Frame
Merge Task AnimationCloud Processing
IoT Devices in the Wild
Monitoring / Analytics
Decision Making
Changing the Environment
Cloud Processing
EOD Report
Trading Report
Market Data
Company Transactions
Financial Models
Many scenarios, fewer patterns Big Data Big Compute Microservices at Scale All built from basic elements
Cloud Scale Patterns
BigData
Big Compute
Your goal is to run a workload / service Workload breaks into pieces
Compute Storage Networking
Unit of work Manageable Recoverable
Building Hyper Scale Apps – Scale Units
Local vs. Blob VHD Storage Custom Images vs. Gallery Images Scaling up vs. out Patching/ security story Azure throttling limits
As you design, consider:
Cloud Services
Deploying Resources with Azure Resource Manager
Resource Groups
Collection of resources with a similar lifetime
Every resource belongs to a resource group
Resources have types, defined by resource providers
RBAC integration Declarative model driven
deployment Consistent
RESOURCE GROUP
• Re-usable components
• Simple Orchestration
• Simple Roll-back
• Source file, checked-in
• Specify resources, dependencies and connections
• Parameterized
Instantiation of repeatable config.Configuration Resource Group
Azure Templates - Repeatability
SQL - A Website VirtualMachines
SQL-AWebsite[SQL CONFIG] VM (2x)
DEPENDS ON SQLDEPENDS ON SQL
SQLCONFIG
http://azure.microsoft.com/en-us/documentation/templates/
Resource Group
Subnet
Storage
VM
VNET
Public IP
Azure Resource Manager: Building a Virtual Machine
NIC storageAccount- accountType
publicIPAddress- allocationMetho
d- domainNameLa
bel
virtualNetwork- addressSpace- Subnet
- addressPrefix
networkInterface- privateIPAlloc
ationMethod
virtualMachine- hardwareProfile- osProfile- storageProfile- networkProfile
https://github.com/Azure/azure-quickstart-templates/tree/master/101-simple-windows-vm
Azure Resource Manager: Deploying an App
Resource Group
Subnet
Storage
VM
VNET
Public IP
Resource Loops: Declare multiple resources
NIC
https://github.com/Azure/azure-quickstart-templates/blob/master/resource-loop-vms-vnet/azuredeploy.json
VM
Public IP
NIC
VM
Public IP
NIC…
Subnet
Storage
{ "apiVersion": "2015-05-01-preview", "type": "Microsoft.Compute/virtualMachines", "name": "[concat('myvm', copyIndex())]", "location": "West US", "copy": { "name": "virtualMachineLoop", "count": "10" }, "dependsOn": [ "[concat('Microsoft.Network/networkInterfaces/', 'nic', copyindex())]", "[concat('Microsoft.Storage/storageAccounts/', parameters('newStorageAccountName'))]" ],
Resource Loops: copy() / copyIndex()
{ "apiVersion": "2015-05-01-preview", "type": "Microsoft.Network/networkInterfaces", "name": "[concat('nic', copyindex())]", "location": "West US", "copy": { "name": "nicLoop", "count": "10" }, ...
For each resource, use copy to set the number
Use copyIndex to create the names.
1
2
Deploying a Resource Loop
Define multiple storage accounts
Allocate VMs per storage account depending on workload
Load balancer routes connections to individual VMs
Scaling to 100s of VMsResource Group
VNet
Subnet
Load Balancer
Storage Account
40 VMs
40 NICs
Storage Account
40 VMs
40 NICs
Storage Account
40 VMs
40 NICs
Public IP
Traffic Manager -> Multiple load balancers
Multiple VNETs VMs run workloads in 1
or more containers Event Management
across application Container repository in
Azure Auto-scale containers
and VMs
Larger scale example
Containerization Abstract the unit of
work from the VM Lightweight, fast
provisioning, density
Simplify application updates
Works with cluster managers for auto-scale capabilities
Container Host - 1
ContainerA
ContainerA
ContainerB
Container Host - 2
ContainerA
ContainerA
ContainerB
Orchestration Engine
ARM Template2 Hosts
4 Instances of Container A2 Instances of Container B
Manage groups of identical VMs
Auto-scale
Performance
Customization
Ease of Management
Future VM Scale Sets: “Next-Gen Worker Role”
https://github.com/gbowerman/azure-myriad/
Resource Group
Subnet
Scalable Storage
VM
VNET
Scalable NIC…
VM
VM
VM
Scale Set
Extensions
"networkProfile": { "networkInterfaces": [ { "namePrefix": "vmssNic", "properties": { "ipConfigurations": [ {
"publicIPAddressNamePrefix": "vmssIp", "subnet": { ...
VM Scale Sets: targetInstancecount
{ "type": "Microsoft.Compute/virtualMachineScaleSets", "apiVersion": "2014-12-01-preview", "name": "myVMSS", "location": "East Asia", "Properties": { "scaleSettings": { "targetInstanceCount": "40" }, ...
Specify a target instance count
Specify a prefix for repeating resources.
1
2
Virtual Machine Scale-Sets
How this all fits together
Groups of containers
Groups of VMs
Application Orchestrators (e.g. Mesosphere)
Your Application/Service
Making Life Easier
VMs and VM Scale Sets
Azure Public CloudAzure Stack Private Cloud
VM Extensions
SCALR, RightScale, Mesos, Swarm
Service Fabric (VMs and
Containers)Batch
App Service Media
Web Apps
MobileApps
Apprenda, CloudFoundryOpenShift, Jelastic, Marathon
Infrastructure
IaaS and IaaS+
General Compute
PaaS
Vertical Compute
PaaS
IaaS v2: Open Choice at Every Layer
Cluster Orchestratio
n
HDInsight
Why higher level services
• Commodity to high-performance• Small to huge• RDMA network for
tightly coupled tasks
• IaaS or Cloud Service• Standard or
custom images• Windows or Linux
Cloud ServicesIaaS
Virtual Machines
Hardware
User workload or service
Get and manage VM’s
Start the tasks
Move task input and output Queue tasks
Install task applications
Scale up and downTask failure? Task frozen?
Manage and authenticate users
Significant amount of infrastructure, not related to business
Hadoop Clusters in Azure (with Yarn) Big data analytics Deploys MapReduce (MR/Hive), Real time
streaming (Strom) and NoSQL database (HBase) applications
IaaS offering now in preview Up to hundreds of nodes
HDInsight
Designed to code at scale in Azure Highly variable workloads Installs applications Gets the task on the machine Up to thousands / tens of thousands of
nodes
Batch
Batch Demo
Offer video transcode as a service
Used by MSN, XBox, Office Video, Skype, and others
Scale Unit – 1 video Their focus is video – so
they use Batch
Example: Media Services
Azure Batch
Azure Media Services
Batch Demo
You can build anything on Azure Choice at every level of the stack Azure Resource Manager Templates help
scale repeatable resources Scale Sets will simplify auto-scale Higher level services (Batch, HDInsight etc.)
make things easier while you focus on the app logic
Summary
For more informationSession Title Speaker(s) When
BRK3505 Taking a Deep Dive into Microsoft Azure IaaS Capabilities
Drew McDanielMahesh Thiagarajan
Tues, May 59:00AM - 10:15
BRK3705 Running Large-Scale Batch and High Performance Computing Applications with Azure Batch
Karan Batta Wed, May 61:30PM - 2:45
BRK3478 Deploying and Managing Services with Microsoft Azure Service Fabric
Chacko Daniel Wed May 65PM-6:15
BRK3722 Managing Linux and Windows on Microsoft Azure with Chef Kundana Palagiri Thu, May 7
9-10:15 AM
BRK3702Running Docker Containers on Microsoft AzureSpeakers:
Khalid Mouss, Tom Hauburger
Thu, May 710:45 AM - 12:00
BRK2557 Harnessing the Power of Hadoop: Cloud Scale with Microsoft Azure HDInsight
Lance Olson Tues, May 510:45 AM – 12:00
Thank you
Visit Myignite at http://myignite.microsoft.com or download and use the Ignite Mobile App with the QR code above.
Please evaluate this sessionYour feedback is important to us!
© 2015 Microsoft Corporation. All rights reserved.