BizTalk roadmap and Biztalk 2016 (Sam Vanhoutte @ Codit's BizTalk 2016 Launch)
Microsoft BizTalk Server End-to-End Performance Testing Guidance Ewan Fairweather & Paolo Salvatori...
-
Upload
jayden-pickup -
Category
Documents
-
view
224 -
download
3
Transcript of Microsoft BizTalk Server End-to-End Performance Testing Guidance Ewan Fairweather & Paolo Salvatori...
Microsoft BizTalk Server
End-to-End Performance Testing GuidanceEwan Fairweather & Paolo SalvatoriBizTalk Customer Advisory TeamMicrosoft
Session Objectives
• Describe how we run BizTalk performance assessments
• Demonstrate tools and techniques that can be used during a BizTalk performance assessment
• Introduce the BizTalk Server Performance Optimization Guide
Session Agenda
• Assessing application performance• Key learnings• BizTalk Server performance
assessment−Methodology−Tools
Tuning the Microsoft Platform
Using LogParser To Generate KPIs For Your Results
Example results spreadsheet from a recent performance lab
Demo
Session Agenda
− Assessing application performance− Key learnings • BizTalk Server performance
assessment−Methodology−Tools
Key LearningsPlatform Tuning
• Turn off hyperthreading• Adjust Windows Server 2003 for best
performance• Use a separate disk for PAGEFILE• Stop unnecessary services (for
instance, Print Spooler)• Defrag volumes hosting BizTalk dbs
Key LearningsPlatform Tuning• Disable the SynAttackProtect feature• Prevent antivirus scanning• Networking
− Adjust TCP/IP parameters in the registry− Use 1-gigabit networks− Separate subnets for Mgmt, Data, and
Messaging− Tune NICs for performance
− Memory for packet buffering− Offload processing to NIC if possible
Key LearningsPlatform Tuning – CLR Hosting Threads• To prevent thread starvation, BizTalk needs
enough threads allocated to the .NET thread pool• Controlled by per host reg key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$hostname\CLR Hosting
DWORD entry Default Value Recommended Value
MaxIOThreads 20 100
MaxWorkerThreads 25 100
MinIOThreads 1 25
MinWorkerThreads 1 25
1 2
3 4
Key LearningsBizTalk Host Configuration • Split up the sending, receiving, processing,
and tracking into multiple hosts • This technique allows each host to:
− Have its own set of resources such as memory, handles, .NET thread pool
− Decreases the contention on the MessageBox Host Queue tables
• Avoid running too many host instances on the same box
Key LearningsSQL Performance• Isolate BizTalk data and transaction log files• Pre-allocate space to prevent autogrowth at
run time• Create multiple data files of the same size,
one for each available core on the SQL machine, for the TempDb
• Add filegroups and files to the BizTalkMsgBoxDb
• Scale up the SQL Server resources− CPU, disk, memory
• Separate BizTalk Databases across servers• Multiple MessageBox databases• Configure DTA Purge and Archive job
Key LearningsLow Latency• Internal message queue• MaxReceiveInterval parameter
− 500 ms default
• BatchSize parameter• Turn off global tracking• Use inline sends
1
23
Key LearningsOrchestrations (1 of 2)• Eliminate unnecessary persistence points• Minimize the internal state/size• Use the Call Orchestration shape for performance• Asynchronous mechanisms give flexibility but
require a MessageBox round trip− Direct Bound Ports, Self-Correlating Ports and Start
Orchestration shape
• Eliminate logical ports and use inline sends• Prefer distinguished field over promoted properties• Turn off unnecessary tracking:
− Message send and receive tracking− Begin and end tracking shape
• Define a performance counter for each orchestration type
Key LearningsOrchestrations (2 of 2)• Avoid using XmlDocument for processing large
messages− The memory amount required to load an
XMLDocument is up to 10 times the actual size of the message
• Instead use one the following techniques:// Use an XmlReader object to read the content of the message partusing (XmlReader reader =(XmlReader)xlangMessage[0].RetrieveAs(typeof(XmlReader))){
...}// Retrieve the content of the message part as a stringusing (string message =(XmlReader)xlangMessage[0].RetrieveAs(typeof(string))){
...}// Deserialize the content of the message part to an instance of a custom .NET classusing (RequestClass request =(XmlReader)xlangMessage[0].RetrieveAs(typeof(RequestClass))){
...}
1
2
3
Key LearningsPipelines (1 of 2)• Use PassThruReceive and PassThruTransmit
pipelines whenever possible• Avoid loading the entire document into
memory using an XmlDocument object− Use a streaming approach instead
• If you need to read the value of specific nodes inside an XML message part, do not use the SelectSingleNode method exposed by the XmlDocument class− Use an instance of the XPathReader class contained
in the Microsoft.BizTalk.XPathReader.dll assembly− Use an instance of XPathMutatorStream class
contained in the Microsoft.BizTalk.Streaming.dll if you need to read and eventually change the value of one or more elements.
Key LearningsPipelines (2 of 2)• Use an instance of the VirtualStream class
(Microsoft.BizTalk.Streaming.dll) when reading a large message
• Promote items to the message context only if you need them for:− Message Routing (Orchestrations, Send Ports)− Demotion of message context properties (Send Ports)
• “Write” to the context instead of “Promote”• Use the IPipelineContext.ResourceTracker to track
and dispose non-CLR resources• Instrument your source code to make your
components simple to debug• Avoid ordered delivery and transaction support
Optimizing BizTalk Memory Usage
Streaming techniques that can be used in pipelines
Demo
WCF Adapters• Whenever possible, use one of the following WCF-* Adapters.
− WCF-NetTcp Adapter − WCF-Custom Adapter + CustomBinding with the
binaryMessageEncoder and tcpTransport Binding Elements.
• Add the serviceThrottling service behavior to a WCF-Custom Receive Location and use the following settings:
Parameter Default Suggested tcpTransport->ConnectionPoolSettings->maxOutboundConnectionsPerEndpoint
1 try changing to 20
tcpTransport->maxPendingAccepts 1 try changing to 20 tcpTransport->maxPendingConnections 10 try changing to 20 tcpTransport->listenBacklog 10 try changing to 20
Send Side (BizTalk Send Port):
Receive Side (BizTalk Receive Location):
Parameter Default Suggested tcpTransport->maxPendingAccepts 1 try changing to 20 tcpTransport->maxPendingConnections 10 try changing to 200 tcpTransport->listenBacklog 10 try changing to 20
Parameter Default Suggested serviceThrottling->maxConcurrentCalls 16 try changing to 20 serviceThrottling->maxConcurrentSessions 10 try changing to 20 serviceThrottling->maxConcurrentInstances - leave the default
WCF Adapter
Optimizing WCF for performance
Demo
Session Agenda
− Assessing application performance− Key learnings • BizTalk Server performance
assessment−Methodology−Tools
BizTalk Server Performance Optimization Guide
MSDNhttp://msdn.microsoft.com/en-us/library/cc558617.aspx
Announcing
Assessing Application Performance• The usual story
− We all know testing is important for every solution
− Testing is neglected
• This is a problem− Solutions using BizTalk tend to be
business critical and tolerate little downtime
− Poor testing increases the risk profile of your project
− This runs the risk of huge technical and political problems
Minimize Project Risk
• Unit testing• Functional testing• Integration testing• User acceptance testing• Disaster recovery testing/dry runs• Overload testing• Performance assessment
Effective testing
reduces your project risk
Session Agenda
− Assessing application performance− Key learnings • BizTalk Server performance
assessment−Methodology−Tools
BizTalk Performance Assessment• An end to end process with a set of
goals and objectives
• Occurs over a set period of time• It requires a team of individuals with
the right skill sets to “tune the stack”
• Iterative knowledge transfer processOptimizeOperateDesign & Deliver
POC Performance Assessment Pre-Live
Performance Assessment
Post-Live Performance Assessment
ScopeEngagement Summary• Determine resource requirements
− What scenarios are you testing− What technologies are involved− What adapters are needed− SAN requirements− 32 bit vs. 64 bit
• Other requirements− Throughput− Latency
• Document all information!− Guidelines, Goals, Constraints, etc.
• Staff the right people
ScopeDefining Goals• Provide objectives and success criteria for
the lab• Goals should cover
− Performance metrics− Throughput, Latency
• Roles and Responsibilities• Deliverables• Define any constraints
− Is hardware fixed?− Scale up/out BizTalk/SQL− Can third party systems be scaled out?
− Web servers, MQ Series, Custom DB’s
ScopeDefining Goals (Example)Objective
Measure the Maximum Sustainable Throughput (MST) for the given use cases with different hardware configurations.
Test Case 1Messaging-Only scenario
Throughput : 50 messages/sec sustainableLatency: response time < 2 sec
for 90% of messagesTest Case 2
Payment service BizTalk orchestration Throughput : 3.6 million messages withinan 8 hour period (125 messages/second)Latency : N/A
ScopeLogical and Physical Architecture
1
23
4
PlanDetermining Resource RequirementsTimeline
− 2 weeks of preparation time− 1 week per test case− 1 week to document findings
SMEs required− MQSeries, SQL Server,
IIS, SAN Specialist
Hardware request− SAN Storage (HBA, Luns), Networking, Clustering
Lab space− Appropriate space, whiteboard, projector, security of
room
PlanSample Timeline
1 2 3 4
5
6
Prepare
• Detailed Solution Design• Code complete• Invest in test assets
− Build/Deploy− MSBUILD, MSI,
BTSTask, Scripts− Functional test
− BizUnit, Visual Studio− Load test
− Loadgen, Visual Studio
Building The Lab EnvironmentInstallation and Configuration• Infrastructure
− RAID10/0+1?− Databases and logs separated− Database autogrowth− Virus scanning software
• Third party systems− Non-default values, scale-out options
• BizTalk− Host configuration− Message tracking− Adapter configuration
Building The Lab Environment• Performance monitoring• Deployment
− MSBuild, BTSTask, MSI
• Functional testing• Validate automated load test• Document Solution
Performance Baseline− Latency− Throughput− Spool depth− etc
Executing the Performance Lab Assessment High Level Flow• Run automated test
− BizUnit, VSTS, LoadGen
• Document results− KPIs, Check after every run
• Tuning for performance − One modification at a time− Perform platform
modifications
Session Agenda
− Assessing application performance− Key learnings • BizTalk Server performance
assessment−Methodology−Tools
Helpful Tools
• BizTalk BPA 1.2• PerfMon• BizTalk LoadGen• BizUnit• Log Man• Relog• Log Parser• PAL• SQLIO • BizTalk Orchestration Profiler
Related Content
• MSDN Guidance− Microsoft BizTalk Server Performance Optimization Guide
http://msdn.microsoft.com/en-us/library/cc558617.aspx − Microsoft BizTalk Server Operations Guide
http://msdn.microsoft.com/en-us/library/cc296643.aspx − BizTalk Server 2006 R2 Hyper-V Guide
http://msdn.microsoft.com/en-us/library/cc768518.aspx
• BizTalk Customer Advisory Team Site− http://msdn.microsoft.com/en-us/biztalk/dd876758.aspx
Resources
• BizTalk Server 2006: Managing a Successful Performance Lab− http://msdn2.microsoft.com/en-us/library/aa972201.aspx
• Scaling Your Solutions− http://msdn2.microsoft.com/en-us/library/aa561567.aspx
• Configuration Parameters that Affect Adapter Performance− http://msdn2.microsoft.com/en-us/library/aa561380.aspx
• Persistence and the Orchestration Engine− http://msdn2.microsoft.com/en-us/library/aa547090.aspx
• Troubleshooting MessageBox Latency Issues− http://msdn2.microsoft.com/en-us/library/aa561709.aspx
• Performance Tips and Tricks− http://msdn2.microsoft.com/en-us/library/aa560694.aspx
• Rule Engine Configuration and Tuning Parameters− http://technet.microsoft.com/en-us/library/aa548008.aspx
• Performance Considerations When Using the Rule Engine− http://msdn2.microsoft.com/en-us/library/aa952929.aspx
Filegroup – Resources• BizTalk Server Database Optimization
− http://msdn.microsoft.com/en-us/library/bb743398.aspx
• Understanding Files and Filegroups− http://msdn2.microsoft.com/en-us/library/ms189563.aspx
• How to: Add Data or Log Files to a Database (SQL Server Management Studio)− http://msdn2.microsoft.com/en-us/library/ms189253.aspx
• How to: Move an Existing Index to a Different Filegroup (SQL Server Management Studio)− http://msdn2.microsoft.com/en-us/library/ms175905.aspx
• Physical Database Files and Filegroups− http://msdn.microsoft.com/en-us/library/ms179316(SQL.90).aspx
• Working with tempdb in SQL Server 2005− http://www.microsoft.com/technet/prodtechnol/sql/2005/working
withtempdb.mspx
• Optimizing Server Performance Using I/O Configuration Options− http://msdn2.microsoft.com/en-us/library/ms189892.aspx
MQ Adapter Resources
• HOW TO: Set Up and Test the BizTalk Adapter for MQSeries− http://support.microsoft.com/kb/821489
• BizTalk Adapter for WebSphere MQ− http://msdn2.microsoft.com/en-us/library/aa772116.aspx
Q & A
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED
OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.