Susan Windsor Insight Through IntelligenceWMHL Consulting Limited, MDTitle slideFuture for Functional Test Automation?TM Forum April 2006

AGENDA

Future for Functional Testing Challenges for Functional AutomationStrategic DirectionWhat does this mean to you?

Future for Functional Testing

Increased Testing Demand..?Global development, with large projects having multi site, multi geography and multi suppliers to contend with.Corporate and regulatory requirements growing all the timeBusiness demands new applications, faster and cheaper to obtain competitive advantageCloser alignment of IT and business to repair lack of business confidence

Or, increased competition for roles? Business testers using frameworks reduces the number of traditional functional testers requiredAgile development methods mean developers undertake more unit and component testingGrowth of outsourced testing to different geographies

Is the role of the UK functional tester (who is neither technical nor business specialist) dead?With the focus on skilling up functional testers, are we fiddling whilst Rome burns?

Challenges of Functional Automation

Record and Playbackall you need to do.!Easy to record the scriptsExtremely fragileExpensive to maintainOn average, each test run requires at least 50% of the scripts to be recorded again

Please tell me no one does this anymore!!!

ScriptingUse the scripting language to write scripts that do what you wantBuild in as much robustness as possibleBut, youre building an application to test an application, which also needs testing!Maintenance costs can be very highNeeds programmer skills Cost of implementation not affordable at project levelChallenging for Test Managers to incorporate in overall test plans

Table DrivenLike scripting but more flexible and greater re-useRemove items from the script that change, e.g. dataReduces maintenance costs a bitImplementation costs about the sameStill requires specialist skills to implementStill a challenge for Test Managers to control

Functional Test Automation is Broken!Focus on technology rather than business needs80% of functional testing still manual60% to 70% of automation tools used for non-functional testingTypically, traditional functional automation stops at 100 scripts, regardless of test coverage requirementCritical factorsCost of implementation and maintenance prohibitiveInsufficient and expensive skills required Inability to asset share over different technologies

Project Structure also a challenge

Strategic Direction

Automation FrameworksWouldnt it be good if..Tests could be documented in common format, regardless of whether they are manual or automatedThe format for the tests resulted in quicker preparation time than traditional manual testsBoth developers and business testers could understand and use the same test formatScript maintenance was no longer a requirementTests could use any of the test execution tools required by the underlying technology

Faster planning, faster execution, and far less technical skill required

Business Analysts already using them, and use will growHome grown frameworks built within organisations to meet business demandsNiche suppliers providing frameworks; try a Google search 512,000 hits this weekSeen a handful that appear matureLatest review by Paul Herzlich (OVUM analyst) Market Leaders such as Mercury developing Business Process Tester (BPT) This is the industry direction now

Where Frameworks Fit

Frameworks & Industry NeedsIts too early to automate in our projectDefine the test cases in the Framework now, and automate laterWe havent got manual testing under control yet!Use the Framework as the standard test case definition and use for manual and automated scripts We do Agile development, functional automation doesnt fitDefine test cases before the development commences and use the documentation to obtain business signoff against requirements, as soon as code is written, you can automate

Frameworks & Industry Needs - 2Automation consultants are too expensive to increase our automation test coverage the way wed like toYou can fully utilise your manual testers to build test cases to generate automated scripts, your automation specialist can concentrate on the more exciting aspects of test automation and multiple projects can be support by a single technical resource. Our offshore partners use automation but we are concerned at the lack of visibility over exactly what their tests doFull test documentation can be generated to demonstrate what the automated scripts will do so that you retain control

Frameworks & Industry Needs - 3We are tied into our Automation Supplier and even though were not comfortable with that, we cant see a way outTransfer your test case definitions into Framework now, initially prove your test cases via interaction with existing tools and you can switch when ever you likeWe cant afford the cost of automated testing on this projectUsing a Framework will reduce the cost of automation implementation by between 58% and 75% over traditional approachesWe find the cost of maintenance prohibitive to extend our test automation furtherUsing a Framework will reduce the cost of automation maintenance costs by between 75% and 85%

What Does This Mean to You

Discussion TimeWhat is your organisations direction?What is your view on Automation Frameworks?Do they have a future?Do you require different skills?What is your future?

Susan Windsor WMHL Consulting Limited, MD

susan@wmhl.co.ukClosing slideThank You