Augmenting Coded UI

47
13 LogiGear Corporation. All rights reserved. Augmenting Coded UI with Action-Based Test Automation Microsoft | LogiGear

Transcript of Augmenting Coded UI

Page 1: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Augmenting Coded UIwith Action-BasedTest AutomationMicrosoft | LogiGear

Page 2: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Test Requirements for Quality Enablement

10x

Speed

MaintainabilityReusability Scalability

Page 3: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

- Quickly and easily create a large number of tests

Test Requirements for Quality Enablement

Speed

10x

Page 4: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

- Quickly and easily create a large number of tests

- Maintain tests when the application or GUI changes

Test Requirements for Quality Enablement

Speed

10x

Maintainability

Page 5: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

- Quickly and easily create a large number of tests

- Maintain tests when the application or GUI changes

- Focus on reusable test design and flow, not test programming

Test Requirements for Quality Enablement

Speed

10x

MaintainabilityReusability

Page 6: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

- Quickly and easily create a large number of tests

- Maintain tests when the application or GUI changes

- Focus on reusable test design and flow, not test programming

- Execute data-driven testing for coverage and scalability

Test Requirements for Quality Enablement

Speed

10x

MaintainabilityReusability Scalability

Page 7: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

- Quickly and easily create a large number of tests

- Maintain tests when the application or GUI changes

- Focus on reusable test design and flow, not test programming

- Execute data-driven testing for coverage and scalability

- Enable test understanding and team collaboration

Test Requirements for Quality Enablement

Speed

10x

MaintainabilityReusability Scalability

Page 8: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Action-Based Testing Methodology

Keyword-driven methodology

Modular test design

Solution architecture and

functionality

Automation Engine

ActionLibrary

Interface

Test ModuleTest ModuleTest Module

Test Development Plan

Page 9: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

• The test developer creates tests using "actions“

• Each action consists of a “keyword” and “arguments”

def action_NewProduct . . . .

def action_AddQuantity . . . .

def action_CheckQuantity . . . .

Keywords and Actions

number name quantity

new product P-9009 Sledge Hammer5

number quantity

add quantity P-9009 20add quantity P-9009 3add quantity P-9009 6

number quantity

check quantity P-9009 34

© 2013 LogiGear Corporation. All rights reserved.

Page 10: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Low-level action: mouse Click

Group of low-level actions: Select a product Click “Add to Cart” Click “Buy”

High-level action: Checkout.

Higher-level action: Order Entry

Low-level actions …

… combined to make higher-level

actions

… combined to make higher-level actions

© 2013 LogiGear Corporation. All rights reserved.

Action-Based Testing Methodology

Keyword-driven

Page 11: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Original functionality was OKNew functionality is SUBMIT

Every instance of the changedaction is automatically updated

Tests are easily maintained when application requirements change

Action-Based Testing Methodology

Reusability

© 2013 LogiGear Corporation. All rights reserved.

Page 12: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

CHECKPAYMENT

CLOSEAPP

RENTCAR

Action Library

Custom Actions

STARTAPP

TEST MODULE Car Rental

user key

TEST CASE TC 01 Rent Cars

first name last name car

last name amount

FINAL Exit App

window

start app sam

rent car Sam Doe Ford Escape

check payment Doe 89.95

close app reservation

PROCESSORDER

CONFIGACCNT

ADDINS

CHANGEORDER

LISTCARS

Create tests with reusable Actions

PRINTRECEIPT

Easy-to-use test creation for readable, understandable test flows

CHECKPAYMENT

Page 13: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

… results in only a small increase in additional actions

2000tests

200actions

20 functions

A large increase in test cases …

4000tests

250actions

22 functions

Reusability enables scalability, for high-quality tests

Page 14: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Cash FlowOptimization

ReconcileStatements

Management

Image Verification

Payment SystemAuthorization

Authentication

Access Mgmnt

Access

Secu

rity

Risk/FraudMitigation

CrimeDetection

TransactionMonitoring

Data Comm

Tran

sactio

ns

Real-time Processing

Action keywords need a methodology …

Test Development Plan

Business

Objective

Business Objective

Business

Objective

Business ObjectiveBusiness Objective

Business

ObjectiveBusiness Objective

Business Objective Business

ObjectiveBusiness

ObjectiveBusiness Objective

Business Objective

Cash Flow

Optimization

Reconcile

StatementsManagement

Image Verification

Authorization

Authentication

Access MgmntRisk/FraudMitigation

CrimeDetection

TransactionMonitoring

Data Comm

Real-time Processing

Page 15: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Action-Based Testing Methodology

Keyword-driven methodology

Modular test design

Solution architecture and

functionality

Automation Engine

ActionLibrary

Interface

Test ModuleTest ModuleTest Module

Test Development Plan

Page 16: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Modular Test Design

Objective-driven process at the Business-object level

Test Development Plan• Top-down Test Development Plan

Page 17: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Objective

Test Module

Objective

Test Module

Objective

Test Module

Modular Test Design

Test Modules, not Test Cases, are the main testing component

Test Development Plan• Top-down Test Development Plan

• Business-level Test Module objectives

Page 18: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Test ModuleTest ModuleTest Module

TEST

CASE

Modular Test Design

The best method to scale automation to 100’s or 1000’s of test cases

Test Development Plan• Top-down Test Development Plan

• Business-level Test Module objectives

• Objective-driven Test Cases with similar “scope”

TEST

CASE

TEST

CASE

Page 19: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Test Module

TC

Modular Test Design

Test Development Plan• Top-down Test Development Plan

• Business-level Test Module objectives

• Objective-driven Test Cases with similar “scope”

• Action Library and Interfaces drive test case operation

Test Module

TC

Test Module

TC

ACTIONLIBRARY

INTERFACE

Page 20: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Modular Test Design

• Top-down Test Development Plan

• Business-level Test Module objectives

• Objective-driven Test Cases with similar “scope”

• Action Library and Interfaces drive test case operation

• Automation engine to execute Test CasesSeparate test development and test automation

environments

Test Module

TC

Automation Engine

Test Development Plan

Test Module

TC

Test Module

TC

ACTIONLIBRARY

INTERFACE

Page 21: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

• Suited for complex, enterprise-level test projects

• Test modules, with readable, understandable test flows

• High team involvement, collaboration and communication

• Emphasis on test design, test flow and test project organization− Less focus on tools, technology, and

programming details

Maintainable, Reusable and Scalable Test Automation

Focus on test design and test flow, not on test programming

Test Module

TC

Automation Engine

ACTIONLIBRARY

INTERFACE

Test Development Plan

Test Module

TC

Test Module

TC

Page 22: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Cash FlowOptimization

ReconcileStatements

Management

Image Verification

Payment SystemAuthorization

Authentication

Access Mgmnt

Access

Secu

rity

Risk/FraudMitigation

CrimeDetection

TransactionMonitoring

Data Comm

Tran

sactio

ns

Real-time Processing

Action keywords need a methodology …

Test Development Plan

Business

Objective

Business Objective

Business

Objective

Business ObjectiveBusiness Objective

Business

ObjectiveBusiness Objective

Business Objective Business

ObjectiveBusiness

ObjectiveBusiness Objective

Business Objective

Cash Flow

Optimization

Reconcile

StatementsManagement

Image Verification

Authorization

Authentication

Access MgmntRisk/FraudMitigation

CrimeDetection

TransactionMonitoring

Data Comm

Real-time Processing

Page 23: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Cash FlowOptimization

ReconcileStatements

Management

Image Verification

Payment SystemAuthorization

Authentication

Access Mgmnt

Access

Secu

rity

Risk/FraudMitigation

CrimeDetection

TransactionMonitoring

Data Comm

Tran

sactio

ns

Real-time Processing

Automation Engine

ACTIONLIBRARY

INTERFACE

Security Business-level

Objectives

Transaction Business-level

Objectives

AccessBusiness-level

Objectives

Test Module Test Module Test Module

Test Development Plan

Action-Based Testing: Organized, Modular Test Design

Page 24: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Action-Based Testing Methodology

Keyword-driven methodology

Modular test design

Solution architecture and

functionality

Automation Engine

ActionLibrary

Interface

Test ModuleTest ModuleTest Module

Test Development Plan

Page 25: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Action-Based Testing Solution

Test Design

TestObjectives

Test Lines(with Actions)

TestModules

ActionDefinitions

InterfaceDefinitions

CustomScripts

Test Development Test Automation

Test and Action

Repository

Architecture and functionality designed for Action-Based Testing

Integrated in the Quality Enablement process

Extension in Microsoft Visual Studio 2012, and TFS/MTM Continuous Integration environment

Page 26: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Action-Based Testing Solution

Architecture and functionality designed for Action-Based Testing

Integrated to the Quality Enablement process

Extension in Microsoft Visual Studio 2012, and TFS/MTM Continuous Integration environment

source: Microsoft, with permission

Page 27: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Action-Based Testing Solution

Architecture and functionality designed for Action-Based Testing

Integrated to the Quality Enablement process

Extension in Microsoft Visual Studio 2012 and TFS/MTM Continuous Integration environment

Team Foundation Server

Microsoft Test

Manager

Visual Studio

Page 28: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Fragile Test Scripts

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

Test Scripts

ApplicationUnder Test

50Quantity

use nailsDescr

123Number

hammerName

Add

OK

Test Scripts are fragile when linked directly to the application

Page 29: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

50Quantity

use nailsDescr

123Number

hammerName

Add

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

Test ScriptsFragile Test Scripts

OK

ApplicationUnder Test

When application code changes, test scripts require maintenance

Page 30: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

INTERFACE Add Product

Name ID TA Name

ORDER_Button 34 OK

50Quantity

use nailsDescr

123Number

hammerName

Add

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

Test CasesStable and Maintainable Test Cases

INTERFACEApplicationUnder Test

Test Cases are linked to a user-defined logical name in the interface

Page 31: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

INTERFACE Add Product

Name ID TA Name

ORDER_Button 34 OK

50Quantity

use nailsDescr

123Number

hammerName

Add

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

TC n

src ctrl

click add OK

Test CasesStable and Maintainable Test Cases

INTERFACEApplicationUnder Test

Interface is linked to the control properties of the application

Page 32: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

number name description quantityrow 57463292 Tires Radial tires 8row 45892135 Axles Between the wheels 10row 32198327 Wheels 17 inch wheels 24row 32014827 Taillights Red plastic 326row 03512305 Headlights For nighttime driving 22

name criterion

filter high quantity quantity > 8filter low quantity quantity < 100

number name description quantityrow 57463292 Tires Radial tires 8row 45892135 Axles Between the wheels 10row 32198327 Wheels 17 inch wheels 24row 32014827 Taillights Red plastic 326row 03512305 Headlights For nighttime driving 22

name criterion

filter high quantity quantity > 8filter low quantity quantity < 100

name

use data set /Inventory Data Set

number name description quantity

add product #number #name #description #quantity

name number description quantity

check product #name #number #description #quantity

repeat for data set

number name description quantityrow 57463292 Tires Radial tires 8row 45892135 Axles Between the wheels 10row 32198327 Wheels 17 inch wheels 24row 32014827 Taillights Red plastic 326row 03512305 Headlights For nighttime driving 22

name criterion

filter high quantity quantity > 8filter low quantity quantity < 100

Data-driven Test Cases

Multiple data sets applied to a single Test Module for code coverage

Test Module

TESTCASE

TESTCASE

TESTCASE

TESTCASE

Page 33: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Visual Studio 2012 Test Environment

Team Foundation Server

Visual Studio IDEMicrosoft Test Manager

Unit testing

Performance testing

Load testing

Code quality analysis

Test planning and

management

Test case manageme

nt

Lab Manager

Manual testing

Exploratory testing

Work item tracking Source control Lab management

servicesTest execution

services Build services Reporting and BI

UI automatio

ntesting

source: Microsoft, with permission

Page 34: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Test Case Association

Test Case

Visual Studio 2012 UI Test Automation

Coded UI Test Authoring

Manual Coding

Import Existing

Test Case

Record Actions

Execution Methods

Direct Executio

n

Build Integration

Test Manager

Visual Studio IDE

Unit testing

UI automation

testing

Performance testing

Load testing

Code quality analysis

Defect Reporting Quality Analytics

Code Coverag

e

Build Reports

Bug Creation

Keyword

Based

source: Microsoft, with permission

Test Case Association

Test Case

Page 35: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

• Connect to TFS• Define MTM tests• Build and check-in

tests into TFS• Associate the tests

to MTM tests• Run the tests in MTM

and examine the results

Microsoft TFS / MTMContinuous Integration

© 2013 LogiGear Corporation. All rights reserved.

Page 36: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Automatically interact with the app like a user would

Follow steps your team has recorded—faster, unattended, continuously

Coded UI Example

© 2013 LogiGear Corporation. All rights reserved.

Page 37: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Convenient Record & Playback easily replicates user actions to automatically generate test code

Coded UI tests are run as part of the Continuous Integration automated build

C# code defines methods, params, and property details

Coded UI tests run as part of the unit testing

suite

Coded UI tests in the familiar Solution

Explorer, are a part of source control

Coded UI Example

© 2013 LogiGear Corporation. All rights reserved.

Page 38: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Keyword-driven example

© 2013 LogiGear Corporation. All rights reserved.

Page 39: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Keyword-driven example

© 2013 LogiGear Corporation. All rights reserved.

Page 40: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Keyword-driven example

© 2013 LogiGear Corporation. All rights reserved.

Page 41: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Keyword-driven example

Add one plane to cart

© 2013 LogiGear Corporation. All rights reserved.

Page 42: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Create a high-level action:add plane to cart

Keyword-driven example

© 2013 LogiGear Corporation. All rights reserved.

Page 43: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

High-level action:add plane to cart

Keyword-driven example

© 2013 LogiGear Corporation. All rights reserved.

Page 44: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.

Identical keyword actions and Coded

UI functionality

Keyword-driven example

Lower-level programming details are hidden from the

test flow

Page 45: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Scalable, reusable Action-Based Testing methodology

Readable, understandable

test flow

add plane to cart

Re-use the same high-level action multiple times

© 2013 LogiGear Corporation. All rights reserved.

Page 46: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Augmenting Coded UIwith Action-BasedTest Automation

A single, integrated solution for:

• Test Development – fast, easy-to-use

• Test Management – reusable, maintainable

• Test Automation – scalable, customizable

www.TestArchitect.com

© 2013 LogiGear Corporation. All rights reserved.

Page 47: Augmenting Coded UI

© 2013 LogiGear Corporation. All rights reserved.

Thank you !