From Unit Testing to Load Testing : Visual Studio 2010-Your Complete Testing tool

19 Aug 2011

SANDEEP JOSHIAVP Technology, Barclays Capital

@sandeepmvp http://blogs.msdn.com/sajoshi

Agenda Developer’s Role in Quality Options for automated tests Automated Functional Testing

Coded UI Test Demo

Load Testing− Web Performance Tests− Load Tests− Load Test Rigs− Performance Explorer

Developer’s Role in Quality

− Find Problems before you make them− Code Analysis− Code Metrics− Performance Tests

− Don’t let bugs out of your sight− Unit Testing and Code Coverage− Test Impact Analysis− Coded UI Tests

− Or into your builds− Gated Check-In

Dose Of Reality

− You know you should work that way.− You probably don’t.− This talk is for you.− You have problems.− How do you figure out what’s going

wrong?

It’s not that difficult as you think.

Options for automated tests Several kinds of automated tests

DatabaseBusiness Logic

User Interface

DatabaseUnit Tests

T T T T T

Unit Tests

T T

T T T

T T

Web Performance

Tests

T

T

T

T

Coded UI Tests

T

T

Automate Functional Testing

70%of all functional testing is still done manually.

Automate Testing - Coded UI Tests

− Automate functional tests.

− Record actions or import action recording from test.

− Easily repeatable.

Coded UI Test

demo

Load Testing:

Disclaimer: I’m going to use the term “load testing” in most general sense.

Load Testing – Goals

− Identify the capabilities of your application

− Possible goals− Explore− Verify − Find the limits− Crush

The Tools

Visual Studio

Ultimate

• Web Performance Tests• Load Tests• Load Test Rigs

Visual Studio Premiu

m

• Profiling Tools• Performance Wizard

Disaster Strikes.

− The application is deployed to production.

− Users are losing their minds.− Intermittent errors on the site.− Site is slow.

− Your boss is seriously angry.− “Fix it!”

The Problem.

− “No Repro”− “Works on my box.”− You don’t know what the errors are.− The site seems to work fine.

− Wild guess: it dies under load.− You need to create some load.

The Plan.

− Script user activity with Web Performance Tests

− Simulate a mix of users using Load Tests

− Simulate lots of users with a Load Test Rig

− Reproduce the errors...hopefully− Find code problems with the

Performance Wizard− Fix the code

What is a Web Performance Test?− Test type in Visual Studio 2010

Ultimate− Simulate a user using an ASP.NET

application− HTTP traffic

− Building block of Load Tests

Tip: Web Tests & User Stories

− Web Tests should simulate a User Story

− Example:− As an administrator, I need to search for a

person by username so that I can deactivate that person’s account.

− As a customer, I want to order a pizza and pay with a credit card.

− Helps to organize your suite

Data-driven Web Tests

− Run web test once per row in data source

− Simulate multiple users doing *similar* actions

− Data source types− OLE DB, CSV, XML

− Parameterization− Better tests, better coverage

What are Load Tests?

− Mix of Web Performance and Unit Tests

− Simulates a mix of users doing a mix of things

− Simulate − Network speeds− Different browsers− Varying user activity loads

Unit Test in a Load Test?

− Helpful for testing WCF− Cumbersome to simulate WCF via

Web Test− (You’d have to re-invent the wheel.)

− Unit tests can re-use the WCF libraries− (Done and done.)

− Important for Silverlight and SOA− http://wcfloadtest.codeplex.com/

− http://sqlloadtest.codeplex.com/

Demo

− Creating and Running Load Test with VS2010

Two ways to run load tests.

− Generate load from Visual Studio 2010− Uses only 1 core/processor− Limited to 250 simulated users

− Generate load from a Test Rig− Far better test− Simulate lots more users

Load test Rigs & Application

Web Performance

Tests

T

T

T

T

T

T

T

T

T

T

T

T

T

T

T

T

Test Agent

Test Agent

Test Agent

Test Agent

Test Controlle

r

ControllerOrchestratesLicensingPerfMon Counters

Agent (1..n)Simulates users

Application Under Test

Application Under

Test

Generates Load

Coordinates &

Gathers Informatio

n

Controller

Agent 1 Web Server

DatabaseAgent 2

Agent n

Load Test Agent Licensing

− Microsoft Visual Studio Team System 2010 Test Load Virtual User Pack

− Simulate up to 1000 virtual users− Used to cost $$$

Visual Studio 2010 Load Test Feature Pack− Used to cost ~$3500.00 per 1000

simulated users− Free starting March 8, 2011− Visual Studio 2010 Ultimate with

MSDN− Simulate unlimited amount of load

− (Limited only by your hardware.)− http://www.microsoft.com/

visualstudio/en-us/products/2010-editions/load-test-virtual-user-pack

Profiling Tools

− Load Tests help you find issues in a system− “Surface” issues

− Profiling helps you find issues in the code

− Unit Tests, Web Tests, Load Tests− Repeatable− Good for targeted tuning

− System under “natural” load− Not repeatable− Looking for rough patterns

Profiling Methods• Gathers information at intervals (clock

cycles)• No code modifications• Use this for first pass explorations

Sampling

• Modifies your code• Gathers detailed timing and count

information

Instrumentation

• Multi-threaded code profilingConcurrency

• Object creation• Garbage collection.NET Memory

• Interaction between your application and SQL Server via ADO.NETTier Interaction

Additional Resources

Visit Microsoft’s website:www.microsoft.com/visualstudio/test

Channel 9:http://channel9.msdn.com/learn/courses/vs2010

Team Blogs:http://blogs.msdn.com/amit_chatterjeehttp://blogs.msdn.com/vstsqualitytoolshttp://blogs.msdn.com/edglashttp://blogs.msdn.com/lab_management

Feedback

9

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Visual Studio, the Visual Studio logo, and [list other trademarks referenced] are trademarks of the Microsoft group of companies.

 The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.  Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the

part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.  MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.