Lessons Learned from Migrating Legacy Enterprise Applications to Microservices
Migrating Customers to Microsoft Azure: Lessons Learned From the Field
-
Upload
ido-flatow -
Category
Technology
-
view
162 -
download
2
Transcript of Migrating Customers to Microsoft Azure: Lessons Learned From the Field
![Page 1: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/1.jpg)
Migrating Customers to Microsoft Azure: Lessons Learned From the
FieldIdo Flatow
Senior Architect, Sela GroupMicrosoft MVP & RD
Level: Intermediate
![Page 2: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/2.jpg)
Azure From the Field• Azure Costs and Billing• Know Your Azure Stuff• Architecture Lessons Learned• Tools of the Trade
![Page 3: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/3.jpg)
AZURE COSTS AND BILLING
![Page 4: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/4.jpg)
Azure Costs – the Obvious• Pay for the services you use
– VMs, Storage, SQL Database, Service Bus, …• Pay only for outgoing bandwidth• Pay-as-you-go• Don’t remember the prices? Check online
http://azure.microsoft.com/en-us/pricing• Don’t forget discounts
![Page 5: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/5.jpg)
The Costs You Didn’t Account For• Small change
– VM disks, bandwidth, storage transactions, VPN gateway• The must-have
– VM Backups, SQL Database backups (exceeding 200%)• The nice-to-have
– Monitoring data, diagnostics data– Where are the data ingestions points?
• Don’t go to production without it– Technical support
![Page 6: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/6.jpg)
Reducing Azure Costs• Which storage type are you using? (prices in $/GB/month)
LRS (0.024) / ZRS (0.03) / GRS (0.048) / RA-GRS (0.061)• Turn off dev/test VMs off-hours (can reduce cost by 50%)
– Azure Automation– Auto-shutdown feature for v2 VMs– Azure Dev/Test Labs
• Keep track of your VM instances– Delete unused VM disks– Use basic tiers for dev/test– Delete staging deployments
![Page 7: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/7.jpg)
Optimize Usage by Monitoring• Azure provided services
– Azure Portal’s service metrics– Microsoft Operations Management Suite (OMS)
• SaaS– https://www.cloudyn.com– https://azure-costs.com – https://www.datadoghq.com – https://www.cloudability.com
• Regularly inspect your bill !
![Page 8: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/8.jpg)
HOW STUFF WORK
![Page 9: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/9.jpg)
Learning How Azure Works• None of us know how Azure really works• Except maybe this guy
![Page 10: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/10.jpg)
Learning How Azure Works• Learning how stuff works is a process
– Start with the basics of what each Azure service does
– Get to know each service’s limitations and performance
– Look for best-practices, advanced topics, etc.• Here are some examples of the learning curve
![Page 11: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/11.jpg)
How Stuff Works - Lessons Learned• Load Balancer
– Azure LB is round robin, not!– Is LB the same for all hosting offerings?
• Traffic Manager– What’s DNS load balancing?– TTL is a recommendation
• Azure Compute– Persistent/Temporary disk– Does size only dictate CPU & Memory?
(short answer, NO!)– Incoming/outgoing IP
![Page 12: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/12.jpg)
Where to Learn Azure Services• Get updated
– Azure blog (http://azure.microsoft.com/blog)– Team blogs(Azure Storage, Active Directory, ScottGu)– Service update (http://azure.microsoft.com/en-us/updates)
• Know your limits– Service limits and quotas (http://bit.ly/1mNOSlp)– Storage scalability (http://bit.ly/1aPZNt4)
• Check the documentation– Walkthroughs (http://azure.microsoft.com/documentation)– ARM REST APIs (http://bit.ly/1WWboMN)– Azure p&p (http://bit.ly/1XqW2yo // http://bit.ly/1jTXUje)
• Virtual/Classroom training, Azure user groups (https://www.meetup.com/find/?keywords=azure)
![Page 13: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/13.jpg)
LESSONS LEARNED: ARCHITECTING FOR THE CLOUD
![Page 14: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/14.jpg)
Azure Scenarios• Lift & Shift • Dev/Test• PaaS
– Hosting– *-as-a-Service
• Internet of Things (IoT)• Big Compute / Big Data• Backup / Disaster recovery
![Page 15: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/15.jpg)
Cloud Architecture Best Practices• Scale Up / Scale Out• Mind your state• Disaster Recovery (DR) will help you• DIY vs. Azure PaaS Services• Decouple components with queues
• This is the classic “Azure Architecture Best Practices” sessionhttp://bit.ly/24YZlNOhttp://bit.ly/1Yz5y3a
![Page 16: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/16.jpg)
Lessons LearnedWhy Can’t I Access my Service?• Complaints
– Traffic Manager marks services as degraded (failed)– Azure Load Balancer “ignores” instances– External monitoring with Pingdom fails due to timeouts
• Troubleshooting– Checked Azure operation logs, but did not find anything substantial– Requested Azure Support to provide logs for TM / LB,
but they didn’t have detailed logs for all services– Azure Support requested IIS logs, but we didn’t have
any (deployment already deleted/updated)– Time spent so far – about 4 days!
![Page 17: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/17.jpg)
Lessons LearnedWhy Can’t I Access my Service?• After enabling logs we realized that:
– TM monitoring got blocked by VM Endpoint ACL– Endpoints failed when probed due to an internal error – HTTP 500 in IIS
Logs– Endpoints hanged (>1 minute) when probed because site was overloaded
• Conclusions– Keep in mind that ACL Endpoints also blocks Azure IPs– Don’t assume Azure monitors everything for you– Don’t rely solely on Azure Metrics for diagnostics– Collect application, server, and IIS logs– Azure Storage may not be the best place to store logs
(unless it’s connected to an HDInsight cluster)
![Page 18: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/18.jpg)
Lessons LearnedMaking Developers Happy• Complaints
– Deploying to cloud services takes forever– Deployment sometimes hangs, and we need to delete it and
re-do• Why is that?
– Deployment means uploading huge packages to storage
– Complex worker roles are slow to stop / start– Provisioning new instances takes time
![Page 19: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/19.jpg)
Lessons LearnedMaking Developers Happy• So what can we do?
– Use compute emulator when possiblehttps://docs.microsoft.com/en-us/azure/vs-azure-tools-emulator-express-debug-run
– For slow Internet – deploy from an Azure VM– Enable Web Deploy for single-instance Web Roles
https://msdn.microsoft.com/en-us/library/azure/ff683672.aspx– Don’t be afraid to RDP and replace .dll files (dev/test)
http://blogs.msdn.com/b/kwill/archive/2013/09/05/how-to-modify-a-running-azure-service.aspx
![Page 20: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/20.jpg)
Lessons LearnedData Synchronization and DR• We know how to deploy frontends and backends for DR:
– Deploy multiple sets of Web and Worker apps to multiple sites– Use configuration to manage database and storage addresses– Create Traffic Manager endpoints with failover
• We don’t know how to properly:– Replicate the database between the sites– Replicate Azure Storage (blobs, tables, files) between the sites
![Page 21: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/21.jpg)
Lessons LearnedData Synchronization and DR• Database Synchronization
– (IaaS) SQL Server 2014/2016 – AlwaysOn supports HA and DR with multiple datacentershttp://blogs.msdn.com/b/igorpag/archive/2014/12/22/sql-2014-high-availability-and-multi-datacenter-disaster-recovery-with-multiple-azure-ilbs.aspx
– (IaaS) MySQL – Use Master-Master and Multi-Master replicationshttp://top-performance.blogspot.co.il/2013/09/mysql-multi-master-replication.html
– (PaaS) SQL Database – Active (readable) Geo-Replication in all tiershttps://docs.microsoft.com/en-us/azure/sql-database/sql-database-geo-replication-overview
![Page 22: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/22.jpg)
Lessons LearnedData Synchronization and DR• Azure Tables synchronization
– Like with databases, active-active is hard to achieve– Use RA-GRS storage, for a secondary readable table– For non-secondary regions, consider a transactional data
pattern• Write transactions (commands) to a topic, not directly to the table• Workers dequeue transactions, and write them to the table• For DR, use identical subscriptions with workers in remote region• Always use version checks to detect concurrency/race issues
![Page 23: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/23.jpg)
Lessons LearnedData Synchronization and DR
• Blob / File synchronization– Azure doesn’t support storage synchronization -
DIY– Use Azure Functions to automate replication– https://cmatskas.com/copy-azure-blob-data-betw
een-storage-accounts-using-functions– Standard blob trigger polls a container – For large containers (>10k blobs), notify a
container has changed by using queues
![Page 24: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/24.jpg)
Lessons LearnedAzure Services are not Bulletproof• Complaints
– Azure Storage sometimes timeout when reading/writing– SQL Database connections sometimes timeout or dropped– SBQ suddenly isn’t available for several seconds
• What is happening?– Some Azure services use shared computing,
networking, and storage– Network congestion and high CPU can
cripple a single service node– Potentially, such interruptions should not
take more than a few seconds
![Page 25: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/25.jpg)
Lessons LearnedAzure Services are not Bulletproof• Resolving with retries
– Use fault detection and retry logic– Most Azure SDKs support retries (Service Bus, Storage)
https://azure.microsoft.com/en-us/documentation/articles/best-practices-retry-service-specific
– For other cases – use the Transient Fault Handling Application Blockhttps://msdn.microsoft.com/en-us/library/hh675232.aspx
• Resolving with failovers/backups– Service Bus secondary namespace
https://msdn.microsoft.com/en-us/library/azure/jj554355.aspx– Read-access GRS for Azure Storage
https://msdn.microsoft.com/en-us/library/azure/dn727290.aspx
![Page 26: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/26.jpg)
Lessons LearnedMaking it Faster• Complaints
– Upload to Azure Storage is slow– It takes a long time to send 10k messages to Service Bus Queues– Listener in Worker Role only utilizing 25% CPU
• Azure services are complex, don’t expect <1ms latency• What can we do?
– Do basic tests to get a baseline, make sure it is reasonable– Read performance guidance articles – many services have them– Investigate, investigate, investigate!!!
![Page 27: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/27.jpg)
Lessons LearnedMaking it Faster - Examples• Azure Storage
– Disable Expect-100 and Nagle– For large files, use parallel block uploads– http://azure.microsoft.com/en-us/documentation/articles/storage-perfor
mance-checklist
• Service Bus– Reuse clients and factories– Batch messages when sending and receiving– Client-side batching and server-side prefetching– https://
azure.microsoft.com/en-us/documentation/articles/service-bus-performance-improvements
![Page 28: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/28.jpg)
Lessons LearnedMaking it Faster - Examples
• Database– Improve SQL on IaaS by using temporary SSD and disk
stripinghttps://msdn.microsoft.com/en-us/library/azure/dn133149.aspx
– Improve SQL Database by using Active Geo-Replication for load-balancinghttp://azure.microsoft.com/blog/2014/07/12/spotlight-on-sql-database-active-geo-replication
![Page 29: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/29.jpg)
TOOLS OF THE TRADE(EVERYTHING BUT VISUAL STUDIO)
![Page 30: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/30.jpg)
Azure Management Tools• Azure Management Studio
http://www.cerebrata.com/products/azure-management-studiohttp://www.cloudportam.com
• IaaS Management Studiohttp://iaas.ao-is.com
• Service Bus Explorerhttps://github.com/paolosalvatori/ServiceBusExplorer
![Page 31: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/31.jpg)
Azure Storage Tools• Azure Storage Explorer (cross-platform)
http://storageexplorer.com • CloudBerry Explorer
http://www.cloudberrylab.com/free-microsoft-azure-explorer.aspx
• CloudXplorerhttp://clumsyleaf.com/products/cloudxplorer
![Page 32: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/32.jpg)
Command Line and SDKs• Command-line
– PowerShell cmdletshttps://github.com/Azure/azure-powershell
– Azure CLIhttps://github.com/Azure/azure-cli (CLI 2.0)https://github.com/Azure/azure-xplat-cli (CLI 1.0)
– Resource Manager templateshttps://azure.microsoft.com/en-us/documentation/templates
• Service Management API– Azure Resource Explorer
https://azure.microsoft.com/documentation/articles/resource-manager-resource-explorer – Azure SDK for XXXX
(Where XXXX is: Net, Java, Python, Node, Go, PHP, Ruby) https://github.com/Azure?q=azure-sdk-for
![Page 33: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/33.jpg)
To Conclude• Having a cloud mindset is important• Architecting for cloud is how we do it properly• Don’t say “I though it worked differently” – read,
learn, try!• With time and experience, new best practices
are published • Keep up with the technology pace
![Page 34: Migrating Customers to Microsoft Azure: Lessons Learned From the Field](https://reader036.fdocuments.us/reader036/viewer/2022081520/586f72a01a28ab10258b54a1/html5/thumbnails/34.jpg)
Resources• You had them throughout the slide deck• Here’s another link
https://visualstudio.com/vs/azure-tools • My Info
– @IdoFlatow – [email protected] – http://www.idoflatow.net/downloads