So You Think You Can Write a Test Case - XBOSoft Webinar
Transcript of So You Think You Can Write a Test Case - XBOSoft Webinar
XBOSoft, Inc. All Rights Reserved. 1
So You Think You Can Write a Test Case
Webinar with Srilu Pinjala #WriteaTestCase
XBOSoft Dedicated to software quality improvement
Founded in 2006
We speed products to market with our expert:
• Software QA consulting
• Software testing
Global team with offices in San Francisco, Beijing and Amsterdam
“Thorough, accurate and fast”
XBOSoft, Inc. All Rights Reserved. 2
House Rules Participants other than the speakers are muted
Questions via the GoToWebinar control on the right side of your screen or through Twitter @XBOSoft
Questions may be asked throughout the webinar - we’ll try to answer them at the end
You will receive info on recording after the webinar
XBOSoft, Inc. All Rights Reserved. 3
Meet Our Speakers
Philip Lew CEO and Founder , XBOSoft • Relevant specialties and
passions o Software quality process,
evaluation, measurement and improvement
o Software quality in use / UX design
o Mobile User Experience and usability
o Cycling and travel
XBOSoft, Inc. All Rights Reserved. 4
Karla Smith Marketing Manager, XBOSoft • Emails you about our
industry hot topic webinars
• Invites you to download our latest trends in software testing whitepapers
XBOSoft Team
Srilu Pinjala
XBOSoft, Inc. All Rights Reserved. 5
• QA professional for over ten years • Expert in testing customer facing applications
• Has worked at Porch, Amazon, IBM • Came up with Green Lantern Automation
Framework where scripts are agnostic to the User Interface
• Presented at PNSQC in 2011 and 2012
• http://linkedin.com/in/SriluPinjala
• http://qasrilu.blogspot.com
• @SriluPNSQC
Why We’re Here
XBOSoft, Inc. All Rights Reserved. 6
1. Because Test cases written during a project are not sufficing for supporting the product through its life.
2. Because test cases are not helping to test the application right or completely.
3. Most test cases’ steps, data, environment, etc. are incomplete or vague.
4. Because new testers are completely dependent on legacy testers for application knowledge, test data, etc.
This is not a webinar for Test Driven Development, Acceptance Driven Development, Testing in agile methodologies, or something new.
Agenda
XBOSoft, Inc. All Rights Reserved. 7
• What is a test case? – What is its purpose? – When to write? – Who is the audience?
• Life Cycle of a Test Case • Key components
– Test Step – Expected Result
• Write it Right – User action – Verification – Test data and Set up
• Test Step types – Test – Navigate
• Sample Test Cases – Typical and revised test
cases – Invoke a URL – Log In
• Test Case with data sheet • Name, Organize, Update,
Maintain #WriteaTestCase
SO YOU THINK YOU CAN WRITE A TEST CASE
Say it like you are talking to a Robot
XBOSoft, Inc. All Rights Reserved. 8
What is a Test Case?
XBOSoft, Inc. All Rights Reserved. 9
Per Wikipedia “A test case, in software engineering, is a set of conditions under which a tester will determine whether an application, software system, or one of its features is working as it was originally established for it to do.”
Why do we need a Test Case?
XBOSoft, Inc. All Rights Reserved. 10
• The products are a result of requirements. • We have been taught to test to verify the
requirements have been satisfied. • For a long time, requirements were used as a
guide or a checklist to test. • So, why can’t we work off of the requirements
and have everything in one document? #WriteaTestCase
Why do we need a Test Case?
XBOSoft, Inc. All Rights Reserved. 11
• Test cases on the other hand are specific instructions to navigate through and to test the product.
• The test instructions and verification cannot be vague or left to assumptions and perceptions.
Purpose of a Test Case
XBOSoft, Inc. All Rights Reserved. 12
“A test case has two purposes: to expose an error, or to demonstrate correct execution.” (Jorgensen, 2014) • It explains when and how to test an
application. • It serves as a guide to learn about the product • It shows the requirements were satisfied. • It exposes errors in case of regression.
Who Are the Users?
XBOSoft, Inc. All Rights Reserved. 13
A. Author of the test cases B. Other testers C. User Acceptance Testers & End Users D. Business Analysts E. Developers F. Project managers G. Anyone on the project H. All of the Above
When to Write a Test Case
XBOSoft, Inc. All Rights Reserved. 14
• It is believed that test cases should be written as soon as the requirement is signed off on.
• It is believed the test cases test the code. • It is believed the test cases should be ready by
the time the first build is ready. • Best time to write a test case – is after the
first round of testing is complete.
After the first round of testing
XBOSoft, Inc. All Rights Reserved. 15
• The interface is available with Pages, elements, data, conditions, etc. which makes the descriptions accurate.
• Working off of requirements limits the test step and expected result to assumptions.
• The test case will serve many QA cycles, not just the first cycle.
• Explore before documenting.
Test Case Life Cycle
XBOSoft, Inc. All Rights Reserved. 16
• Explore the Application under Test (AUT) – code, interfaces, databases, environments, application, etc.
• Document the findings as test cases • Integrate the test cases with different modules of the
application as needed as long test cases or test sets. • Update with test data and environment, etc. • Maintain throughout the life cycle of the product as
it goes through changes.
Test Case Life Cycle
XBOSoft, Inc. All Rights Reserved. 17
Explore
Document
Integrate Update
Maintain
Components of a Test Case
XBOSoft, Inc. All Rights Reserved. 18
Test Suit ID, Test Case Id, Test Case Summary, Related Requirement, Prerequisites, Test Procedures, Test Data, Expected Result, Actual Result, Status, Remarks, Created By, Data of Creation, Executed By, Data of Execution, Test Environment, etc. Are some of the components of a test case as per many online sources.
Key Components of a Test Case
XBOSoft, Inc. All Rights Reserved. 19
Test Step • Test step is user action (if the AUT has a GUI) • Actions to execute, data to input to change the state
of the application.
Expected Result • Expected Result is the changed state of the AUT. • Verification of GUI or Data changes.
Test Step – Dos and Don’ts
XBOSoft, Inc. All Rights Reserved. 20
• Do it right to test it Right. Explain it right. • Keep it clear, simple, specific (CSS) • One step at a time / One action at a time. • Like talking to a Robot. • Don’t instruct “repeat step no. 7 to 10” • Don’t cram multiple steps into one step. • Don’t just say test this. Explain how to test it.
Expected Result Dos and Don’ts
XBOSoft, Inc. All Rights Reserved. 21
• One step can have many verifications. • Describe as many changes and verify them. • Verify page titles, logos, text, images, error
messages, etc. and data especially. • Don’t just say – “page successfully loaded” or
“No Errors”. Explain what defines successful or no errors.
• Like explaining to a Robot.
Write it Right (enable the Tester)
XBOSoft, Inc. All Rights Reserved. 22
• Enable the tester to be able to execute the tests independently.
• Assume the tester knows nothing about the application and instruct them to test right.
• List all necessary steps, data, environment, verification, etc. in the test case.
• Go the extra mile to provide information for correct test execution.
User Action
XBOSoft, Inc. All Rights Reserved. 23
• Type in field something • Select from drop down list something - data • Click button something • Hover over menu something • Go to URL something • SQL - [SELECT userID, userKey FROM Users WHERE userKey = 7]
Verification
XBOSoft, Inc. All Rights Reserved. 24
• Page appears with title something • Header consists of links something,
something • Error message appears – “something” • Dialog box something appears • SQL Verification - userID = [email protected]
Test Data
XBOSoft, Inc. All Rights Reserved. 25
• Test data may be listed in the test step. • It may be listed in a separate column which
will make editing it a lot easier. • Provide SQL queries where necessary. • Maintain Wikis, Excel sheets, etc. for test data • Specify specific data in the test case • Provide steps and tools to create test data
whenever necessary.
Test Setup and Re-setup
XBOSoft, Inc. All Rights Reserved. 26
• Test setup is not a one time thing that needs to happen at the beginning of a test.
• Setup and re-setup can occur many times. • Provide step by step instructions for setup. • Provide enough verification to ensure setup is
as desired for the test case. • Ex- SQL to register > login via GUI > SQL to
update registration date > reset password GUI
Test Environment
XBOSoft, Inc. All Rights Reserved. 27
• Test environment is related to the whole product.
• Companies in this day and age still maintain one test for testing environment compatibility.
• Testing for environment compatibility requires a whole suite of test cases.
• Best to run the whole suite of test cases in every supported environment.
Test Case Types
XBOSoft, Inc. All Rights Reserved. 28
Test • The test steps explore and test the elements. • One user action at a time with one or more
verification criteria.
Navigate • The test steps help navigate to the test element. • More than one user action may be executed as a
navigation function.
Test Case Name
XBOSoft, Inc. All Rights Reserved. 29
Test case name could be divided into parts. Each part indicates a certain aspect of the product.
First part - indicates the product Second part - indicates the module, page, flow Third part - indicates the feature Fourth part - indicates the function
(A test case can have more than four parts)
Test Case Name - examples
XBOSoft, Inc. All Rights Reserved. 30
• Ms.PPT.Invoke • Ms.PPT.mHome_iCut • Ms.PPT.mHome_iCopy • Ms.PPT.mHome_iPaste • Ms.PPT.mHome_iPaste_
oPaste • Ms.PPT.mHome_iPaste_
oPasteSpecial • Col.Search
• Col.nav.Home • Col.Header.lAbout • Col.Header.lFindOffice • Gm.ForgotPassword
Organize
XBOSoft, Inc. All Rights Reserved. 31
Organize • Test execution order • Priority • Dependency
Advantages • Minimized execution time
• Visibility of coverage • Eliminates redundancy
Update or Add
XBOSoft, Inc. All Rights Reserved. 32
Feature change • A new test case may not be necessary. • Update existing test case to include the new steps.
New Feature • One or move new test cases may need to be added.
Ownership and Maintain
XBOSoft, Inc. All Rights Reserved. 33
Owner • The author does not become the owner by default. • A lead, manager, product owner could be the owner
who will oversee the test case through its life cycle.
Maintain • Periodically review test cases and update. • Add new test cases and delete old test cases. • Combine or split test cases for efficient execution.
Remember
XBOSoft, Inc. All Rights Reserved. 34
• Talk to the Robot • Don’t document assumptions and perceptions • Document the specifics • Go the extra mile and enable the tester to be
self-sufficient while executing the test cases • A test case is the guide to the QA, Dev, Prod
environments and the application
Post your questions on Twitter and we'll answer them @XBOSoft Join us to keep updated on all our webinars, reports and whitepapers: facebook.com/xbosoft +xbosoft linkedin.com/company/xbosoft We post regularly on our blog – check us out! http://xbosoft.com/software-quality-blog/ Why not download our free Whitepapers, available here: http://xbosoft.com/knowledge-center/ You will receive an email with information on today’s webinar slides and recording. Any further queries regarding our services or ideas for future webinars please email us! [email protected]
Q+A
www.xbosoft.com