Post on 25-Dec-2014
description
Tester TrainingKevyn Matijevich & Chris Scofield Input by Shahid Khan
Just Break It• Your job is to attempt to break the software.• Think like the user, do not follow the rules.• The requirement follows an order. Mix up the order, when
possible.• Test all browsers, mobile testing, Mac OS testing.• See tips and tricks for more ways to break it.
Types of Testing
White Box
Grey Box
Black Box
• Based on Functional Requirements
• AKA Functional Testing
• Tested by test engineers
• Based on Functional & Design Specification
• Code Based Testing• AKA Unit testing• Tested by Developers
• Based on User Requirements
• AKA User Acceptance Testing
• Tested by Users
Black Box Test Types
Check the integrity of database field
values.
User Interface (aka User Experience) tests how
Users interact and view the software.
Follow the specification step by step and then don’t follow
step by step. Think outside the box, like a user. Don’t JUST follow the test script
path.
Black Box Test Types – cont.
Bounds/Range Checking
Check to make sure a number is within a
certain range. Test the edges and the middle.
Data Validation
Checking that data is valid. UI data matches
DB entries
White & Black Box Test Types
Process intended to reveal flaws in the security mechanisms of an information system that protect
data and maintain functionality as intended.
Due to the logical limitations of security
testing, passing security testing is not an indication that no flaws exist or that the
system adequately satisfies the security
requirements.
White Box Test Types
Installation Testing
Assures that the system is installed correctly and working on the hardware and integrated
modules work together as expected.
Performed to detect defects in the interfaces and interactions
between integrated components or systems, including hardware.
In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the
constructors and destructors. Written by developers as they work on code to ensure that the specific function is
working as expected.
Grey Box Test Types
A subset of all defined/planned tests covering the basic functionality of a component or system. Smoke testing is
conducted in order to ensure that the basic functions of the whole program work correctly, without the necessity to go
into further details. Daily build and smoke testing are common practices.
Tests conducted to verify that the defined user requirements work as the user intended. The tests are usually performed by
clients or end-users. Also referred to as User Acceptance Testing or UAT.
Performance Testing
LOAD/STABILITYTesting heavy loads, such as testing of a web site under a
range of loads to determine at what point the system's
response time degrades or fails.
STRESS
Deliberately intense or thorough
testing used to determine the
stability of a given system or
entity. It involves testing beyond
normal operational capacity,
often to a breaking point, in
order to observe the results.
Executed to determine how a system or sub-system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify
other quality attributes of the system, such as scalability, reliability and resource usage.
Rho’s Agile Methodology• Agile• For Rho, larger project testing is done in an Agile process,
meaning testing is concurrent with software development. As code changes are made, the code is tested manually (soon to be automated) to ensure the code changes meet the requirements that governed them.
Rho’s Other Methodologies
• Few projects at Rho operate under • One phase must complete prior to the
next beginning• Used to be the industry favored
methodology. • Too strict and holds development and
technical advances back
Idea
Analysis
Test
Design
Develop
• Most of Rho’s projects operate under• The planning phase must complete
prior to analysis and the UAT phase must not start until development completes
• Analysis, development, and design may overlap
Waterfall Modified Waterfall
Validation
• Documented evidence that provides a high degree of assurance that software continuously meets the needs of users.
• May be required by Rho when it meets the requirements set forth in Rho’s policies and FDA regulation 21 CFR Part 11
• Validation effort begins with planning, includes all testing types, ends with reporting.
• Short story: define and document user requirements and document testing of requirements.
Role in Validation
Test types commonly used during validation at Rho• Unit• Functional (Regression as well)• Installation (test and production environments)• User Acceptance
Test types commonly used during validation • Unit• Installation qualification (test and production environments)• Functional• User Acceptance• Operational qualification• Performance qualification
Getting Started
Issues completed by Developers Issues vetted by Testers
Test Cases created for each issue (include feature,
release version, functional area, etc.)
Testing executed and issues tracked accordingly
How to Test• A Test Script is compiled by the tester
and added to the Test Cases. This is a list of steps a tester is to complete to verify the validity of the code changes made
• Defects are notated in the Test Case, a new issue is opened for review by the developer if defect does not relate to the active Test Case
• Documentation is critical for testing. See Documenting Your Test card
Test Stack• Through your desktops Remote application, you can access
the following VMs:• RhoVM212- Win7, IE8, Chrome, FF• RhoVM213- Win7, IE10• RhoVM214- XP, IE8, Chrome, FF• RhoVM215- XP, IE7• RhoVM216- Win8, IE11• RhoVM217- Win8, IE10• RhoVM218- Win7, IE11• RhoVM219- Win7, IE9
• Chris Scofield, the Test Engineer is the main contact for gaining permissions to the Test Stack
Tips and Tricks• Multi-click • Data entry tips and tricks• Irregular characters (%, &, #, etc.)• Spacing • Single and double quotes• Cutting and pasting from other software
• Folder for test files (good and bad) Word docs, excel, etc.• Page layouts across browsers• Lorem ipsum website http://www.lipsum.com/
Document Your TestBe explicit in steps for recreationWrite down your inputsTrack your browser and version, operating system
Take screenshots of errors/unexpected outcomesWrite down your outputs
Projects might use JIRA, Crucible, and Zephyr to record test outcomes, others may use
spreadsheets, or a combination of several
Documenting UAT/IQT
A quality control review is
performed on the executed and
completed test script.
Tester follows test steps and documents on test script as detailed in
tester manual and on test
script
Tester is trained how to
document, provided a
Tester Manual to follow, and provided the Test Scripts
Validation Incident Tracking
• SharePoint is used to create and track incidents that occur during validation
• Test Incident Report forms are filled out with:• Status • Incident summary, criticality, priority, corrective action• Type of incident (e.g. tester error, documentation error, software defect)• Resolution Notes, Resolved by, Resolved on• Closed by, Closed on, Closed during software version
Jira Issue Tracking
Agile Development via KanBan board
Bug and issue tracking
Zephyr plugins for Test Case creation
Zephyr JIRA plugin that allows for seamless integration for total Test Management
and data tracking
Creation of Test Cases and Test
Scripts
Creation of test cycles
Test execution tracking and
notes
Regression Testing
• Testing performed after making functional improvement or repairs
• Used to determine if the change has regressed other aspects of the program and introduced new errors.
• Often accomplished through the construction, execution and analysis of product and system tests.
Rho Regression Testing• Rho currently uses a Risk Based approach to Regression
testing. This means that for every Release Version, we only Regression test the features that have code changes. If a feature was not touched, it is not retested. Issues are logged in Excel, sorted by Feature.
• Testing begins with an IQT, then tests are executed on the Test Server (For Agile projects only)
• Execution notes are kept by the testers, all failures are triaged by Project Lead
Communication• Who do you go to for what?• Validation questions
• Kevyn Matijevich• Cheryl Scott• Carrie Dundas-Lucca
• Testing questions• Chris Scofield• Shahid Khan
• Where do you go to for helpful tips?• Testers• Developers• This Training• Internets