Agile Testing - Joe Caravella 1

• From Long Island NY• Spent 20 years in USAF

• Sys Admin• Software Development

• Working in Software Development & Testing since 1993• Hobbies include motorcycles, boating, and water sports

2

• Waterfall – lots of documents and diagrams that become obsolete when coding begins. Plus long development cycles and annual releases

• Scrum-but – follows the basic tenants of Scrum, BUT inconsistent iteration lengths, no retros or reviews, and still doing annual or bi-annual releases

• Scrum – solid understanding and execution of all Scrum principles being led by a certified ScumMaster

• Kanban – workflow queues with WIP limits that encourage “pulling” story cards through your development process and into production

3Agile Testing - Joe Caravella

• My learnings were incremental and continue even today• After many retrospectives and tweaks, I’ll describe some adjustments I’ve made

4Agile Testing - Joe Caravella

• Product, Dev, and QA should sit and work together every day• Rule of Three for all product discussions (QA, Dev & Product)

5Agile Testing - Joe Caravella

• Create a Quality Development Process instead of trying to to test quality into yourproduct

• Unit testing and continuous integration + governance can really help • Get Devs an integrated environment to test their changes before going to QA• Create an automation framework that is dev friendly – they will use it as well

6Agile Testing - Joe Caravella

• The entire development team should create and have buy in to the testing & certification procedures

• Consider the Agile Testing Pyramid

7Agile Testing - Joe Caravella

• The test pyramid is a concept developed by Mike Cohn, described in his book Succeeding with Agile

• Mike’s point was to demonstrate that you should have more lower level unit tests in your testing strategy

• I added percentage goals, exploratory testing, and the need to always try and push testing lower

8Agile Testing - Joe Caravella

• A Mind Map is a diagram used to visually organize information• Exhaustive testing is rarely possible so strategies like Pairwise Testing can help

identify the combination of test cases that have the highest probability to find defects

9Agile Testing - Joe Caravella

• Don’t let your regressions grow so large that it takes more than 1 business day to certify a release

• Testing managers and test leads should add “regression pruning” to their monthly housekeeping procedures

• Querying your previous test results could help identify tests that never fail and are at a lower risk of

10Agile Testing - Joe Caravella

• Testing & development should be parallel activities• Avoid running a testing sprint after a development sprint• Acceptance should occur only after QA is done

11Agile Testing - Joe Caravella

• Avoid making defect fixes directly in QA envs – those change sometimes do not get checked in and the issue will occur in production

• All changes to QA envs should be applied by a versioned increment of your product (this included DB changes!)

• QA should initiate builds & deploys only when Continuous Integration is Green (meaning all UTs have passed)

12Agile Testing - Joe Caravella

• To avoid surprises, use the same people/team that will deploy to Production for all QA deployments

• Many times they will identify an issue that won’t work in Production

13Agile Testing - Joe Caravella

• Create a repeatable release cadence• Don’t sit on code – get value in front of your customers as soon as possible• This will also require teams to figure out how to be more efficient with their

release certification procedures

14Agile Testing - Joe Caravella

• Try to keep production as close to the trunk version of your code as possible• Consider using Release Trains & Feature Toggles

• Release Trains – code needs to be checked in by certain day to make it on the train

• Feature Toggles – configuration that controls when features are available to users

15Agile Testing - Joe Caravella

16Agile Testing - Joe Caravella

• Create a monthly data capture process so you stay informed about changes in your regression

• Try to minimize manual regression as much as possible• If you can keep your automated regression over 85% you are doing very well

17Agile Testing - Joe Caravella

• Create a simple graph of regressions growth and share with teams monthly• This helps teams remain sensitive to what’s being put in regressions• It also helps teams visually see when manual testing is starting to grow too high

18Agile Testing - Joe Caravella

• Identify a way to measure the effectiveness of your quality initiatives• At a minimum, however, an analysis should be performed on issues that escape

the testing phase and becomes customer facing – learnings should be shared across all teams

• Consider creating a company or department KPI that gets updated frequently so teams can react and adjust procedures immediately

19Agile Testing - Joe Caravella

20Agile Testing - Joe Caravella

• It’s difficult to be agile and produce frequent releases using manual regression tests – they just don’t scale well

• When done right, automation helps to create repeatable test execution process with a known timeline

21Agile Testing - Joe Caravella