Post on 13-Dec-2014
description
Effective Testing Practices in an Agile Environment
Raj Indugula raj.indugula@lithespeed.com
www.lithespeed.com
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
Agile Manifesto
Individuals and interactions Processes and tools over
Working software Comprehensive documentation
over
Customer collaboration Contract negotiation over
Responding to change Following a plan over
That is, while there is value in the items on the right, we value the items on the left more.
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
http://www.agilemanifesto.org
3
Shared Context drives Quality
Individuals and interactions
Working software
Customer collaboration
Responding to change
4
Quality is a priority
Agile Approach
Maximize value and quality within specified
project constraints.
5
ROI
Customer Value
Sustainability
Quality
5
Integrated Teams & Iterative Delivery Challenges
6
How do we bridge the communication gap between Bus./QA/Dev? How do we ensure that the evolving software does not regress?
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
Move Quality Upstream
• Testing not a trailing activity
• QA leads the Sprint by providing guidance and feedback on the business needs of the emerging product
8
Testing is Continuous, NOT a Phase
http://testobsessed.com/
9
Testing is Collaborative
• Quality is everyone’s problem, not just of the testers
• Testing is the responsibility of the whole
team
10
Tests represent Customer Expectations
• Shared understanding of what it means for a story to be done
11
Quick Feedback
• Faster feedback loops increase Agility – the ability to respond to change
• Test automation provides rapid feedback on how the software is behaving
12
“Leave No Broken Windows”
• Fix bugs as they are found
• The sooner you find a defect, the cheaper it is to fix
13
It isn't Done until it’s…
“Done Done”
Coded Tested Done
14
Test-Driven
Defining tests with the requirements guides development
http://testobsessed.com/
15
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
ATDD
TDD
Other prac8ces
Agile Engineering Practices
17
Agile Engineering practices allow teams to move fast, be flexible and deliver high quality software
Bill Wake, http://www.xp123.com
What are acceptance tests?
• Tests that demonstrate business intent of
system from end user’s point of view
• Black-box testing
18
What is Acceptance Test Driven Development (ATDD)?
A practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins. - Elisabeth Hendrickson
19
Key Characteristics
1. Elicit details from the business stakeholder(s) about their expectations
2. Distill acceptance criteria into automatable tests expressed in a natural language
3. Wire the tests to SUT with “glue” code (as part of implementation)
20
21
When to Use
• During Requirements Elaboration “First Cut” once the story is written Card level acceptance criteria
• During a Three Amigos session (Business, QA, and
Dev) Expand the story card acceptance criteria Complete set of test scenarios
22
User Story
As a <type of user>, I can <immediate goal> so that
<business outcome>.
User Role (Who?)
Desired Function (What?) End Result (Why?)
23 3 C’s – CARD, CONVERSATION, CONFIRMATION
Modern Agile Acceptance Model
Conditions of Satisfaction – Broad Terms
Acceptance Criteria – Further Refined
Examples – Actual scenarios or data
Executable Examples – Ready to automate
24
Acceptance – Conditions of Satisfaction
Testing a Registration Function • What constraints should we impose? • Business stakeholders and PO agree that
passwords should not be easy to crack.
Please fill in to register.
Email Address
Password
Register
25
Acceptance – Acceptance Criteria
Acceptance Criteria, the Details The PO works with testers and developers from the team, business stakeholders, users or SMEs to come up with this definition of “not easy to crack”: • Must be at least 8 characters and no more than 12 • Must contain only alphanumerics and the period • Must contain at least one digit • Must contain at least one alpha character
Please fill in to register.
Email Address
Password ????????????
26
Acceptance – Examples Written • Examples, Making it Real • Actual examples are the ultimate way to communicate
requirements
Password Expected Expected Message Comment
abc123 Invalid You password must be at least 8 characters long, and no more than 12 characters long.
abcdefghi Invalid Your password must contain at least one character and one number.
1aaaaaaaaa Valid Why valid?
ajx972dab Valid
27
Acceptance – Testable Examples in Gherkin • Executable Example, Making it Repeatable • Examples that can be executed are the final step
Given the “Unregistered User” user has navigated to the “register” page When entering “newuser” in the “Username” field And entering “abc123” in the “Password” field And entering “abc123” in the “Confirm Password” field And pressing the “Register” button Then the text “Thank you for Registering” should appear on page And the URL should end with “use/accountPage”
28
Gherkin – Another Example
• Plain English with a specific format § Feature § Scenario § Given When Then And But
• Can be used for both manual testing and automation
29
Automated Acceptance Tests
Tool (Cucumber, SpecFlow, FitNesse)
Specification expressed in
common language
“Glue” code that ties specification
to SUT
30
Business Analyst Develop usage scenarios
Developer
Create and maintain acceptance test “glue” code that ties test specification to system under test
Tester Create and execute tests to simulate usage scenarios; Automate regression testing
• Should be an integral part of every iteration/sprint, not just a trailing activity
• Vital not only to prove completeness of a feature in repeatable fashion, but also to prove that software does not regress as it evolves
• Success depends on cooperation & collaboration
Testing Collaboratively
31
Benefits of ATDD
• Improved requirements elicitation
• Consensus between BA/QA/Dev on the story helps prevent bugs & gives clear target for development
• Reuse of Acceptance Scenarios for all phases of testing
• Creates clear examples that everyone understands; discovers some problems before any development
32
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
ATDD
TDD
Other prac8ces
What are unit tests?
• Developer tests that determine whether the smallest piece of testable software in an application is behaving as expected
• White-box testing
34
Structure of a Unit Test
@Test!
public void itAddsOnePlayerToTheGame()!
{!
!Game game = new Game();!
!
!Player al = game.AddPlayer("Al");!
!
!Assert.assertEquals(1, game.numPlayers());!
!Assert.assertTrue(game.isPlaying(al));!
!Assert.assertFalse(game.isPlayable());!
}!
Assert
Arrange
Act
35
…all related to same behavior
…title is a behavior
Tests should…
• Not depend on one another • Not affect one another • Leave the system under test unaltered • Be fast executing • Be executable in any order, number or
combination of one another
36
What is Test Driven Development (TDD)?
1. Write a unit test 1. Use unit testing and mocking framework (JUnit, Mockito etc.)
2. May need to write some skeleton code to make it compile
2. Run it and watch it fail (no code yet)
3. Write code until it passes
4. Repeat
37
Write a new test
Red (Failing Test)
Write Code
Green (Test Passes)
The tests help determine what code you write (“Incremental Design”)
Write a new test Test fails
Integrate Write code
Refactor code, make sure tests pass
Test passes
Developer heartbeat
Red, Green, Refactor
Refactoring is supported by the tests and good unit test coverage makes refactoring safe to do
38
TDD – Benefits
• Better design – less coupling
• Supports change
• Prevents gold-plating
• Prevent bugs
• Discover bugs
39
Test Levels
Isolation Tests
• True unit tests, focusing on behaviors
• Many, and fast
Collaboration Tests
• “End-to-end” tests
• Find bugs in conversations between collaborating modules
40
Test Doubles (Mocks)
Used to isolate unit tests form external dependencies
Why use them?
• Ease of setup • Fast executing • To work with a subsystem that doesn’t exist • Simulate various execution paths of external system
41
Test Automation Pyramid
42
Unit Tests/Component Tests
Cucumber, FitNesse, SpecFlow
xUnit, Mocks
Selenium
Developer-centric (Are we building the code right?)
Business-centric (Are we building the right code?)
Adaptation of Mike Cohn's test automation pyramid
Exploratory Testing
GUI Tests
Acceptance Tests
ATDD/TDD Cycle
43
TDD vs. ATDD Historically ATDD did not come about till well after TDD
ATTD § is a specification and testing practice § Determines what “Done” means for a story § Creates a common understanding and shared language
to discuss features (especially with Gherkin) § Cycle times on the order of hours to days
TDD § is a coding and design practice § Provides a mechanism for exploring the solution space § Creates base of tests to support refactoring § Cycle times on the order of seconds to minutes
44
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
ATDD
TDD
Other prac8ces
Version control Tests with Code
Source Code
Unit/Component Tests Acceptance Tests
Source Control Repository
(Git, Subversion…)
46
Requirements gathering Application Development
Source Control
Continuous Integration (CI) Server
Continuously commit changes
and merge changes from others
Pull changes, Build, run tests
Deploy to higher environment
Poll for changes
Development tasks from requirements
test scenarios from business requirements for acceptance criteria
Acceptance Test Environment - Test harness - Tests (automated + manual)
Execute acceptance tests against the deployed application
QA Environment
Deployable artifacts
Build Automation & Continuous Integration
47
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
Barriers to Adoption
Developer : “I’m a developer, not a tester” Management: “Why does Dev help with testing, when we have QA?”
• Necessitates behavioral change • Quality is not free • Simple, but not easy • Requires discipline • Needs generalizing specialists
49
Bridging the Divide
Shared Understanding
Collaborative Testing Quick Feedback
50
“Before beginning a hunt, it is wise to ask someone what you are looking for before you begin looking for it.” —Winnie the Pooh
51
Thanks!
52
Resources
53
Some Tools (open source)
• Unit testing: JUnit, Mockito, EasyMock
• Acceptance testing: FitNesse Cucumber
• GUI testing: Selenium, Sahi
• Javascript testing: Jasmine, Qunit
• Testing the database: DBUnit, SQLUnit, DBFit
• Performance testing: JUnitPerf, JMeter
• Usability testing: Manual unfortunately
54
Some Tools (open source)
• Unit testing: MSTest, NUnit, Moq
• Acceptance testing: SpecFlow
• GUI testing: Selenium
• Javascript testing: Chutzpa
• Testing the database: Tsqlt, SSDT SQL Test
• Performance testing: JetBrains, VisualStudio 2013, SQLServerProfiler, SQLLoadTest
• Usability testing: Manual unfortunately
55
Agile Software Development
• Extreme Programming Explained - Kent Beck • TDD: A Practical Guide – Dave Astels • Refactoring – Martin Fowler • Agile Estimating and Planning – Mike Cohn • Pragmatic Project Automation – Pragmatic Prog.
56
Acceptance Testing
• Bridging the Communications Gap – Gojko Adzic • Agile Testing – Lisa Crispin and Janet Gregory • The Cucumber Book - Wynne M. and Hellesoy A. Gherkin
https://github.com/aslakhellesoy/cucumber/wiki/gherkin Cucumber
http://cukes.info/ https://github.com/aslakhellesoy/cucumber/wiki/
57
TDD/Refactoring
• TDD: By Example - Kent Beck • TDD: A Practical Guide – Dave Astels § The Art of Agile Development – James Shore • Refactoring – Martin Fowler • Working Effectively with Legacy Code – Mike Feathers • Refactoring to Patterns – Joshua Kerievsky • xprogramming.com, refactoring.com, testdriven.com
58