Portfolio Project 3 - Test Plan

Portfolio Project 3

Train2Race

Master Test PlanVersion 1.3

Train2RaceV 3.0

Platform :Microsoft Xbox

Revision HistoryDateAuthorDescriptionVersion

8/1/2015Ryan BateyDesigned test plan.1.0

12/1/2015Ryan BateyWrote up test matrices.1.1

18/1/2015Ryan BateyFinished writing test plan.1.2

ContentsPurpose4I.Target quality and deadlines4Game overview4I. Setup4II. Pickups5III. Front end6IV. Single race6V. Campaign6VI. Settings6VII. Scoring6VIII. Load/save6IX. In-game7Resources7Debug requirements7Test team/contact-info/responsibilities7Assumptions and dependencies9Project risk analysis9Testing priorities and focus9Scope and limitations of testing10Test outline10I. Installation Testing11II. Acceptance Testing11III. Functionality/feature testing11IV. Level checklist12V. Single-player13VI. Front-end and pause menu21VII. Other - soak tests, play-throughs, achievements and unlocks25VIII. Configuration Testing29IX. Verification Testing29X. Regression Testing29Test environment30Patching/build deliveries30Bug database setup31I. Outline of bug reporting31II. Reporting requirements32Debug kits and automation32Problem tracking and resolution33I. Severity and grade of bugs34II. Priority35Software entrance and exit criteria36Sanity testing period36Security and licensing36Glossary/Acronyms36

PurposeThe purpose of this test plan is to test Train2Race in every aspect possible and ensure it is clear of bugs and ready for release within the deadline.The objectives of this test plan are:

To clearly identify each section of the game which is to be tested.

To state the most effective testing approach to carry out for each section.

To produce a checklist for testers to keep track of every detail during the testing period.

Target quality and deadlines:

Train2Race is a triple A title, the target deadline for the testing phase is 1.5 months in duration maximum. Train2Race must be entirely tested and ready for release within a 2 month testing period.

Game OverviewSetup:

Train2Race is a 3d racing game that features 10 cars that can race around 6 race tracks. The game is single-player only, it has no network or multiplayer modes.

Supported Platforms:Microsoft Xbox Only

Supported Controller:Standard Xbox Controller Only

Player Capabilities:1 Player

Multiplayer:Not Supported (including network mode)

Age Rating:All, Everyone

Languages:EFIGS

Territories:TBD, provisionally UK, US and all other EFIGS territories

Pick Ups:

A range of power ups/downs are scattered throughout each track. These are in the form of icons on the track that must be driven over to collect them. Items comprise:

Extra Fuel adds 10 to the fuel counter (which goes from 0-100 internally).

Turbo boost adds 10 to the turbo boost meter (which goes from 0-100 internally).

Oil slick reduces traction for a time, potentially bad for handling.

Spanner repairs any and all damage sustained by the car so far, including damaged tyres.

Tacks causes punctures, permanently degrading traction. Can only be repaired by the spanner pickup.

Missile immediately launches a missile from the car picking this up and that flies straight ahead until it hits something.

Each track should feature at least one of each item.

Front-End

The front-end is rudimentary and only has options for single race, campaign and settings. The hierarchy is as follows:

Single race

- Race selection screen- Track selection

Campaign

- Option to resume campaign (if it was aborted early) or start new, otherwise straight into game.

Settings

Difficulty: easy, medium, hard.Controller: left-handed, right-handed.Language selection: EFIGS (via national flags).Each option screen has a back/cancel option.

Scoring

In single-race mode, ranking is based purely on race position. In campaign mode, points are awarded according to the position attained in each race. The winner of the campaign is the racer with the most points. Points are 10,6,4,3,2,1 for positions 1,2,3,4,5,6 respectively. Between races in campaign mode, there is a single rankings page to display the current ratings.

Load/Save

There are no explicit load or save commands. Progress cannot be saved during a race; choosing to exit a race will lose all information/track position etc. In campaign, career progress to date and the track about to be raced is automatically stored however. When this data is loaded (by resuming a campaign), the player will have the same ranking as before and be placed at the start of the track currently being played.

In-game

There are no in-game options, though when the Start button is pressed, a pause menu appears and the game is halted. Options to return to the main menu, restart current race and resume racing are offered.

Resources:For more information on Train2Race,See pdf document - project_files-GT_portfolio_project_03.

Debug Requirements:Only standard debug kits are to be used by each tester when testing the game.

Each tester is to use standard retail versions only.

Cheats are not to be carried out when testing.

Test Team/Contact-Info/Responsibilities

RoleTester

Name:Contact Details:Mark JonesTel: 0800 123 124 Email: [email protected]

OverviewMark will be the main functionality tester of the project, he will be testing single player as assigned by the test matrix. Each tester is to work in house, and closely with the developers to resolve issues quickly.

ResponsibilitiesSingle player testing as assigned by the test matrixBug reporting using the internal client

HardwarePC#001 Microsoft Xbox #001

LocationLiverpool, UK

RoleTester

Name:Contact Details:John SmithTel: 0800 123 125 Email: [email protected]

OverviewJohn is the projects secondary functionality tester. He will mainly be carrying out soak tests and play through tests. He will also be testing achievements and unlocks. Each tester is to work in house, and closely with the developers to resolve issues quickly.

ResponsibilitiesSoak testingPlay through testing as assigned by the test matrixAchievements and unlocks Bug reporting using the internal client



RoleTester

Name:Contact Details:Robert JohnsonTel: 0800 123 126 Email: [email protected]

OverviewRobert is the projects third functionality tester, he is to test the front end and pause menu of Train2Race. Each tester is to work in house, and closely with the developers to resolve issues quickly.

ResponsibilitiesFront end and pause menu as assigned by the test matrixBug reporting using the internal client



RoleHardware Compatibility Tester

Name:Contact Details:Denise RogersTel: 0800 123 127 Email: [email protected]

OverviewDenise is a third party hardware compatibility tester and will be testing in house with the rest of the test team. She works for a third party testing group that specialises in hardware compatibility testing and will act as the point of contact with them. Denise will be assigned to the project until the testing phase is complete, her main workload is on localised versions of the title to provide a more complete all round localisation testing phase.

ResponsibilitiesHardware compatibility testingBug reporting using the internal client

HardwareThe company Denise works for have provided a suite for her to test in and set it up in house at the developer.


RoleCompliance Tester

Name:Contact Details:James AndrewsTel: 0800 123 128 Email: [email protected]

OverviewJames is a third party compliance tester who will be working in house with the test team. He is to ensure all technical requirements are compliant with the hardware manufacturers guidelines, mainly the official racing bodys requirements as they have right of approval on the finished game. James is to test in house until the testing phase is complete, to ensure all requirements are met legally.

ResponsibilitiesTo ensure the game is compliant with the hardware manufacturers guidelines and all requirements are legally met.Bug reporting using the internal client.

HardwareMicrosoft Xbox #004, #005, #006 and #007.PC#005


Assumptions and DependenciesThe time to turn around each test round will each be a 5 day period at most. Depending on how large an issue is, extra time will be allowed for testers to work with the developers to resolve issues quickly.Each build should be delivered in as short a time period as possible to ensure the game is ready for release within the time schedule.

Project Risk AnalysisEach assigned tester is to be aware of the risks of the project while testing the game. Testers should be aware that the game is to be released to the public with least amount of issues as possible, so that it is to meet the requirements of the developers. These risks include;Is the game fun to play?

Is the game playable as a whole from the consumers point of view?

Does the game meet expectations as it was shown in trailers previous to release?

Does the game include factors to keep the audience entertained all the way through?

All of these are questions each tester should commonly ask themselves while the testing phase is being carried out. Each tester is to put themselves in the consumers shoes so that the game will meet the developers expectations.

Testing Priorities and FocusAll testing is mainly being focused on functionality testing/feature testing for the testing phase. Each tester is assigned to methods of testing by a test matrix for an overall more effective testing phase of the entire game. Play through testing and functionality testing/feature testing are to be covered as thorough as possible by the test matrix in order to save time effectively. The game has been broken down into sections so each tester can navigate the test plan easily. The sections are based on single-player, play through and front end and pause menu. Installation testing and acceptance testing is to be carried out by all testers on the title before all main testing is initiated via the test matrices. Localisation testing training will be provided to all testers before the testing phase is underway and each tester is to use the matrices in the test plan to carry out localisation testing where applicable, during the main testing phase. After all of the main testing has been covered, configuration tests, verification tests and regression tests are to be carried out on the title by all testers.

Scope and Limitations of TestingEach type of testing above is to be carried out within a 1.5 month period. The main focus is so that the play through testing and functionality testing/feature testing phases are carried out specifically, however as long as these types of tests are covered as much as possible soak testing can be carried out. Extra testing will cover much more of the project and lead to a better carried out test plan to ensure quality is assured for the developers.Testing is limited to a lack of console testers within the company, to tackle this issue we will be using a third party compliance tester and also a third party hardware compatibility tester to help to complete the testing phase more efficiently. The hardware compatibility tester is to provide more efficient overall testing for localisation testing, which will lighten the work load and in house testers can focus on testing assigned by the matrix.

Test OutlineInstallation Testing

Acceptance Testing

Functionality Testing/Feature Testing

Level Checklist

Single Player

Front End and Pause Menu

Other - Soak Tests, Play Throughs, Achievements and Unlocks

Configuration Testing

Verification Testing

Regression Testing

Installation Testing

This is the first test to be run on a build, this type of testing must be done by the lead tester/burn mastering technician before duplicating and then passing to the team for further testing. This type of testing should be carried out on every build of the game and run on all relevant platforms. Instructions for testing are as follows Insert the disc in to the Xbox system.

If the game boots, it has passed.

Try to manually boot the game, if it does not boot then it has failed the test.

So long as the game passes the installation tests, acceptance tests can then be carried out on the title.

Acceptance Testing

Acceptance tests are to be carried out to check whether the game is stable enough to be completely tested in QA. This test should be run on every build, this type of testing should be carried out immediately after the installation tests are completed and should take no more than thirty minutes to complete. Acceptance testing is to test each section of the game is functioning correctly, the acceptance test is failed if the following occurs Failure to properly load or execute.

An excessive number of crashes, choppy video or sound or excessive graphical problems.

An excessive number of errors that may lock out parts of the game from testing.

The testing sequence is short and is to verify the game is stable, it is to be carried out by the lead tester as follows Test the game loads up to the main menu.

Test all menus load up - settings, difficulty, controller settings and language selection.

Test single player campaign loads up and the game can be played.

Test single race loads up correctly, through the race selection screen and then track selection screen.

Test all sound plays correctly.

Test all video plays correctly.

Once the game has passed the acceptance tests, the game can be passed to the testing team to carry out proper testing on the title.

Functionality Testing/Feature Testing

Level Checklist

Level:Tested:Not Tested:In Progress:Assigned Tester:Estimated Time:Actual Time:Instructions:Comments:

Level 1

Level 2

Level 3

Level 4

Level 5

Level 6

Single Player

Note: all 6 levels are to be tested in the same manner as the matrix dictates.Level TestedTesterCommentsAdded Test Instructions

Visual

Are the textures displayed correctly?

Are all animations playing correctly?

Any flickering between the textures?

Are all textures blended correctly?

Is there any overlap between graphics?

Are all graphics displayed properly when pickups are acquired?

Are all graphics displaying correctly when overtaking?

Are graphics displaying correctly during a collision?

Text

Is all text displayed on screen correctly?

Is text displayed in the correct selected language on screen?

Does all text match the correct terminology as used in gaming?

Is all spelling and grammar on screen correct?

Are there any words displayed in an incorrect manor?

Does all text have the correct spelling used for each term used in the game?

Does text appear on screen and disappear off screen correctly?

When text is displayed is there any lag or glitching at that specific time?

Audio

Does all music play correctly?

Is there any distortion to music when SFX are played?

Are all SFX played correctly?

When more than one event at a time is carried out are the sound clips playing thoroughly?

Does each song loop properly when its finished?

AI

Does the AI react to the player correctly?

Whilst playing through is the AI behaving as it should?

As a collision happens, does the AI collide correctly?

As an event happens, is the AI glitching in any way?

Camera

Does the camera function correctly when the player attempts to use it?

Is the camera reacting to the players movement correctly?

Does the camera glitch at all when an event occurs?

Is the movement of the camera smooth?

Is there any sticking of the camera when it is in motion?

Does the camera effect the players ability to see what is happening as the game is played?

Options

Does the options menu appear correctly on command?

Are all the consistent menus available within the options menu?

Is it possible to navigate the options menu with ease?

Does bringing up the options menu effect gameplay at all?

Within the options menu, do all settings work correctly?

Is there any lag in the options menu when in use?

Cut-scenes

Do the cut-scenes play through smoothly?

Does a cut-scene initiating effect gameplay?

Can cut-scenes be skipped correctly?

Is there any lag when a cut-scene is played?

Do cut-scenes cause the game to crash?

Are the textures in cut-scenes smooth?

Level specific events (progression)

As the player gains a place on the grid, does the score improve respectively?

Does gaining a place on the grid force any UI errors?

Are there any graphical errors as a place is gained?

When the player overtakes, does the gameplay flow smoothly?

As a pickup is collected, does the UI show any errors?

Do all pick-ups work respectively?

When pick-ups are engaged, does gameplay flow smoothly?

Does collecting a pick-up cause any graphical errors?

As the player earns a new place on the grid, is the score adjusted respectively?

Does the score cause any UI errors as a new score is introduced?

Is gameplay running smoothly as a new score is achieved?

Does achieving a new score effect the games graphics?

Does achieving a new score effect AI correctly?

As the end of the race is met, does the UI respond correctly to scoring?

Is there any graphical errors as the end race scene is played?

Does the gameplay run smoothly as the race is completed?

Is there any obstructions to the race scene playing in full?

Boundary Checking

When more than one button is pressed at a time, does the game freeze at all?

Pushing a button by mistake, does the game react effectively?

When a button is pressed over and over again, is the game caused to freeze?

Does driving towards the barriers/off map cause the game to glitch?

Turning around immediately as the race starts, does the game react properly?

Driving into any other car continuously, does the game glitch at all?

Driving against the barrier, does the player get stuck?

Constantly crashing into the same place on the map, does the player car restart in place properly?

Forcing other cars to crash, does the game react correctly?

Pushing every button on the controller at once, does the game react correctly?

Objects/scenery

Are all objects on the map lined up smoothly?

Is there any blur in textures on the scenery?

Are there any gaps between objects and the scenery?

When crashing are the graphics on the scenery affected negatively?

Can the player interact with the scenery correctly?

Does the AI affect the scenery negatively when interacting with objects?

Do all pick-ups work correctly when acquired?

Is there at least one of each pickup on each map?

Does using a pickup cause the game to jump or lag?

Do pick-ups force the game to run slower?

Front-End and Pause Menu

Note: each tester is to use the above level checklist where necessary.Front-EndTestedTesterCommentsAdded Test Instructions

Does the single race menu appear correctly when selected?

Is there any lag whilst navigating menus?

Does track selection appear correctly when selected?

Is it easy to navigate front-end menus?

Does selecting the menus cause any glitch to the game?

Does the campaign menu appear correctly?

Does selecting campaign cause a glitch to the game?

Does selecting resume campaign appear correctly?

Does selecting start new appear correctly?

Does each selection work as it should once selected?

Settings

In the difficulty menu, is each difficulty selectable respectively?

Can difficulty be changed with ease?

Does choosing a difficulty cause any errors in the menu selection?

Can the controller selections be changed with ease?

When selecting a different controller scheme, do the controls change respectively?

Does each control scheme affect the menus being navigated negatively?

In the language selection menu, can each language be selected with ease?

Does the language change correctly when a new language is chosen?

Are all words on screen in the correct language as chosen?

On each screen, does the back button work correctly?

Does pressing the back button cause any graphical errors?

As the back button is pressed does the game freeze?

In Game Options

Does the game pause correctly when the button is pressed?

Does pausing the game cause any lag?

When the game is paused does any freezing occur?

Does pausing the game affect gameplay negatively?

Does pressing return to start menu work correctly?

When return to start menu is selected, does the game freeze?

Is there any lag when returning to start menu?

Does pressing restart current race cause any freezing?

Does restarting current race affect the gameplay negatively?

Does the game load correctly when restart current race is selected?

When restarting current race, do all scores and positions reset correctly?

When resume race is selected, does the game have any lag?

Does selecting resume race continue the game correctly?

Is there any freezing when resume race is selected?

Does resuming the race affect the score negatively?

Load/Save

Does the game save correctly?

Does the game freeze when saving?

Does saving cause the game to lag?

Does loading the game work correctly?

When loading the game is there any lag?

Is the game being loaded to the correct save file?

When the game has been loaded, was the game saved at the correct place when it was last saved?

Does loading/saving cause the game to function incorrectly?

Other Soak Tests, Play-throughs, Achievements and Unlocks

Achievements and UnlocksTestedTesterCommentsAdded Test Instructions

Once a race is won, is the next race unlocked respectively?

Does the next race unlock when the race is lost?

Does the next race unlock when lower points (than are required to unlock the next race) are achieved in the race?

When the player gains a position in the race are the correct points added?

As a new position is gained, is game flow smooth?

Is there any lag when taking a position in the race?

Does gaining a position have a negative effect on the opposition?

When winning the race, are points added correctly?

Does winning the race cause the game flow to jump or lag?

Is the opposition reacting negatively when the race is being won by the player?

As the player wins the race, do all assets work respectively?

When the player wins the race, is the score adjusted respectively for the opposition?

Playthroughs

Playing the game as a customer would, do all assets seem in place?

Does the game flow smoothly with no bugs?

Is it possible to complete the game?

Can all scores be achieved?

Is the game attractive to play?

Does the game run as it should and play properly?

Playing the game to win, do all scores stack respectively?

Is it possible to win each race in first place?

Does the game finish correctly when the player wins every race?

Playing the game to lose, does the game finish correctly?

Do scores stack properly when losing?

Does losing effect the game flow negatively?

Playing the game and finishing in any other position than 1st, does the game play through properly with no bugs?

Soak Tests

Does the game slow down after being run through the demo mode for a number of hours?

Does running the game through a soak test cause any performance issues?

Is there any memory leaks due to running a soak test?

Does the console stall after a soak test is run?

Is response time in all functions functioning correctly?

Is the game crashing after running a soak test?

Does the game load up correctly after a soak test has been run?

Does the game install correctly after the soak test?

Is the game saving/loading correctly after a soak test?

Are all buttons functioning correctly?

Do all menus navigate smoothly?

Do all cut-scenes run smoothly?

Configuration Testing

Configuration testing consists of running all of the functionality tests that have been used in the test plan to ensure proper functionality on Microsoft Xbox. The test should be run on beta and master candidate discs, testers must use a copy of the functionality test plan to ensure All graphics should be tested to verify that they fit within the TV safe area.

Ensure that the software is tested on all known firmware and hardware releases of the console e.g. all models of Microsoft Xbox. This also includes specific debug machines.

Verification Testing

The purpose of verification tests is to verify that bugs assigned as FIXED, are indeed fixed. Verification tests should be run on every build except those where no bugs are claimed as FIXED. The verification testing is to be executed in this fashion The lead tester will open the QA database and filter all bug reports with the status flagged as FIXED.

The lead tester will then print them out and sort all bug reports by grade, priority, category and the original testers name.

The bugs are then separated by testers name and given to the appropriate tester for verification.

Bugs are to be re-tested by the assigned tester for each bug report.

The bug database is to be updated assigning bugs as VERIFIED FIXED or REOPEN.

Regression Testing

Regression testing is to be carried out regularly, generally once every three builds. It should also be done with all master candidate builds. All testers are assigned to regression testing to verify all issues found in the testing cycle are and have remained FIXED. The primary types of bugs that will need to be tested include: CLOSED, FIXED and REOPEN. Regression testing is to be executed in this manner The lead tester is to print out all bug reports to be regressed.

Separate bug reports by type and tester name. Sort bug reports by grade, priority and category.

Give them to the appropriate tester. Grade A bugs are to be tested first.

The lead tester should then collect all records after they are complete for entry in to the database.

Update the QA database.

Test EnvironmentTesting is to be carried out on standard Microsoft Xbox consoles only, using a standard Xbox controller.

All testing is to be in house at the developer, each tester is to write up bugs into the internal bug reporting client.

Only standard debug kits are to be used by each tester during the testing phase, in certain testing circumstances.

Using standard consoles will ensure the game is tested to a consumers point of view, no debug consoles are to be used as they may have extra memory or a faster processor.

Patching/Build DeliveriesAs a new build is released to be tested, each tester is assigned to use the test matrices as a guide to thoroughly cover each build in turn. Testers will be notified when a new build is integrated into the system, the bug database will close the older build issues making testing much quicker and saving time. Configuration testing is to be run only on beta and master candidate builds, verification testing is to be run on every build except those where no bugs are claimed FIXED. Regression testing is to be carried out regularly on every three builds and should also be run on all master candidate builds. All other types of testing are to be run on every new build released.

Bug Database SetupThe bug database will be the standard reporting client and setup for reporting bugs, the database program is Mantis and will be connected to a server in house for all computers to connect to with ease. Bugs will be grouped into Unassigned, Recently Modified and Resolved groups. Bugs are colour coded according to status e.g. new red, acknowledged yellow and resolved green, each bug will have a unique number that is used to identify issues.When the user clicks the bug number it brings up extended information about that bug, all attached files will be shown such as; screen shots and videos taken by the tester. The user is able to add information to the system about the bug, apply a new status and so on. As a bug is assigned as Fixed by a programmer the tester who originally found the bug will be emailed with its status.

Outline of Bug Reporting

To report bugs each tester is to follow a path of instructions and use the stated programs to describe the bugs clearly.Use Fraps to take screenshots and videos of bugs.

Use Windows Movie Maker to adjust videos for easier pin-pointing of the bugs.

Testers can also use a HD video camera to film gameplay where applicable.

Microphones can also be used to narrate if the issue isnt clear enough.

Here is an example how bugs should be reported in list format, the below template is to be used by each tester:

Assign a bug ID to each bug to keep order for the database, add the intended bug category for each bug below.

Bug ID

Bug AProject

Train2RaceCategory

UI

Description

Write the bug description in short here (example: show steps to reproduce the bug)

Steps to reproduce:

Write the steps up in list format as above.

State how many times the bug has been reproduced and what percentage the bug reproduces at here.

Observed Result

State the observed result of what happens.

Expected Result

State the expected result of what should happen.

Related files

(Any video clips or screen shots should be stated here).

Reporting Requirements

All bugs are to be written up in the above list format. Once a tester has written the bug report, the tester is to send the report to the developer via email address. Each tester will have the phone number and email address of the developers if any issues need to be resolved quickly and at hand.

Debug Kits and AutomationOnly standard debug kits are to be used by each tester, each tester will be running their own personal PC with the debug kits running continuously. Debug kits must keep running for the programmers to sieve out bugs and fix them as required.No automated testing is to be carried out, all testing is to be done manually to save money and time efficiently.

Problem Tracking and ResolutionThe problem tracking and resolution process will be carried out as follows:As soon as a bug is reported and enters the database it is set to NEW and is assigned to the lead tester automatically.

Once the lead tester reviews the bug it is set to OPEN and is then assigned to the producer.

Now the producer assigns the bug to the most relevant member of staff based on the bug type.

The developer who is assigned to the bug has five choices;

NEED MORE INFO will be stated and the bug will be assigned back to the original tester if more information is needed. The tester will then be able to REOPEN the bug.

A FEATURE if the developer believes the bug is a non-issue they may claim the bug as the above. The tester will once again be assigned the bug and decide to state the bug as CLOSED or to report the bug through the chain of management. The Lead tester, QA manager, producer then development director (if the previous cannot decide) will decide whether to state the bug as FIXED, then back to the tester to review the bug as VERIFIED FIXED.

NO FIX if the developer decides the bug cannot be fixed the above will be applied and the bug will be sent back to the tester. The tester can decide whether to assign the bug as CLOSED or to escalate the bug through management and follow through the above steps to state the bug as VERIFIED FIXED.

DESIGNED THAT WAY if the developer decides to state the bug as the above choice, the process as stated before (through chain of management) will be carried out if the tester decides differently. The bug will be assigned to the designer, and the designer will make changes to the design document and then assign the bug to the developer to be stated as FIXED. Once again the bug will be assigned back to the tester to be reviewed as VERIFIED FIXED.

FIXED once a developer fixes a bug the bug is stated FIXED and assigned back to the tester to review the bug. If the tester decides it is not fixed then the bug is stated REOPEN and assigned back to the developer. If the bug is FIXED the tester will state the bug as VERIFIED FIXED and the bug will not be assigned to anyone else, it will be at the end of its life cycle.

Severity and Grade of Bugs

Grade A: a major bug that would prevent the game from being approved or published by QA.A bug that closes or blocks off areas of the game

Any bug that causes the machine to lock up, crash, reset the machine or renders the controller inoperable.

Any hardware compatibility problem that prevents the game from being run on the system requirements.

All copyright and trademark issues.

Content that could contravene the targeted age classification rating.

Any major problem encountered during the installation process that prevents software from being installed successfully.

Non localisation in localised versions in main areas of the game.

Grade B: a serious logic or gameplay defect.Gameplay or UI problems that would hinder the players enjoyment.

Serious graphical corruption or missing artwork.

Localisation spelling errors, such as missing special characters.

Severe game logic problems that would become obvious to the player.

Serious problems caused by the implementation of localised text, audio or graphics.

Audio problems encountered during playback or due to implementation.

Installation configuration and set-up problems.

Localised versions still having non-localised language.

Grammar or spelling mistakes.

Grade C: minor bugs which should be fixed, but is not essential for functionality and release of product.Incorrect implementation of minor functions or features.

Obscure game logic problems that dont cause any side effect (that can be played around).

Inconsistencies between the manual and the software.

Out of context or inappropriate phrasing.

Grade D: gameplay suggestions or improvements recommended by QA that would enhance the products overall quality.Suggestions to improve gameplay or interface.

Additional game design suggestions.

Unintentional game features that would not be noticed by the player unless they had read the original design document.

Issues for the read me file.

Priority

The order in which issues need to be fixed:High

Medium

Low

Software Entrance and Exit CriteriaSoftware is to be tested at the beta, master candidate and alpha stages, on new builds testing will start with quick sanity tests being run. After the previous phase of testing, Train2Race is to be tested overall using the black-box testing strategy. Beginning with installation tests and acceptance tests then on to functionality testing, play-through testing and soak testing. Configuration tests, verification tests and regression tests shall then be carried out, once all above testing is finished then the testing phase is complete.

Personnel Pre-Training NeedsLocalisation testing training will be provided to all in house console testers before the testing phase is under way. We will be enlisting the help of a third party hardware compatibility tester to lighten the load for in house testers. The hardware compatibility tester specialises in localised games testing and will be assigned most of the localisation testing during the testing phase.

Sanity Testing PeriodSanity tests are to be carried out by all three testers when a new build is released to quickly cover the new build and ensure there are no issues left from the previous build.Sanity tests are to be suspended when all previous builds are checked and tested by each tester previous to the black box testing phase.

Security and LicensingThere is a non-disclosure agreement each tester is urged to sign for Train2Race, no information is to be leaked about the game in development and it will be classed as an offence to do so. Microsofts Technical Certification Requirements are to be followed to ensure quality and compatibility of Train2Race with Microsoft Xbox consoles.The third party compliance tester is to ensure the game complies within all of the above rules and especially the official racing bodys rules for the title as they have right of approval over the finished game.

Glossary/Acronyms3d Three dimensional, an object that has height, width and depth, like any object in the real world.Acceptance Testing - acceptance testingis atestconducted to determine if the requirements of a specification or contract are met. Ai Artificial Intelligence, a branch of computer science dealing with the simulation of intelligent behaviour in computers. The capability of a machine to imitate intelligent human behaviour.Alpha Alphais the stage when key gameplay functionality is implemented, and assets are partially finished.A game in alpha isfeature complete, that is, game is playable and contains all the major features.Analysis - Detailed examination of the elements or structure of something.Assets - Assets are classified asimages,multimediaandtextualcontent files.Automation An automatic state of testing, carried out by computers rather than testers.Black box testing - Black-box testingis a method of softwaretestingthat examines the functionality of an application without peering into its internal structures or workings. This method oftestcan be applied to virtually every level of softwaretesting: unit, integration, system and acceptance.Boots When a game loads up on a game system correctly.Bug A defect in a games software or assets.Bug database The database that information about bugs is stored in once a game has been tested and bugs are reported by testers.Bug reporting The approach testers take to write up bugs once a game has been tested.Bug reporting client The software used by testers to report bugs.Build A version of a game that is released to users.Campaign - Acampaignsetting is usually a fictional world which serves as a setting for a role-playinggameor war gamecampaign. Acampaignis a series of individual adventures, and acampaignsetting is the world in which such adventures andcampaignstake place.Cheats A number of button presses or codes that a player can type in so they have numerous ways to exploit a game to their advantage.Checklist - A list of items required, things to be done, or points to be considered, used as a reminder.Choppy Rough or distorted.Collide When two or more objects come into conflict.Collision See collide.Configuration Testing - Configuration testingis the process oftestingthe system with each one of the supported software and hardwareconfigurations.Contact Details of a person who is involved in the game test plan.Crashes When a game freezes and no input by the player changes what the game is doing.Cut-scenes A series of videos played through-out the game.Debug - The process of identifying and removing errors from computer hardware or software.Demo - Demonstrate the capabilities of (software or another product).Developer The person or people that have developed the game.Duplicating To create a second copy of an original.Event When something important happens, like a player gaining a position in a race.Feature Something featured in the game such as a pickup.Fraps A program used by testers to record gameplay and take screenshots.Freeze When a game locks up and is known as frozen.Front-end The main front page of a game that includes all modes to be played and options/settings.Functionality How the game functions and what functions involve.Gameplay How the game generally plays through, usually triggered by a person playing the game.Glitching When a game suffers a sudden malfunction or fault.HD High definition graphics.ID Identification.Implementation - The process of putting a decision or plan into effect; execution.Info Information.Installation Testing - Installation testingis a kind of quality assurance work in the software industry that focuses on what customers will need to do toinstalland set up the new software successfully. Interface - A device or program enabling a user to communicate with a computer.In-game Things that happen while a game is being played are classed as in-game.Integrated With various parts or aspects linked or coordinated.Lag When a game slows down, jumps and skips unintentionally.Leaked When information is given out without lawful consent.Level - Alevel, map, area, stage, world, rack, board, or zone in a videogameis the total space available to the player during the course of completing a discrete objective. The term "level" can also refer to difficultylevel, as in a degree of difficulty.Load A setting in a game that can initiate a previous game save.Localised See localisation.Localisation - Game localisationrefers to the process of transforming videogame software and hardware for preparation to be imported and sold in a new region, usually a different country.Mantis Bug tracking software.Manually - defined as something done by hand or using your physical labour as opposed to something done using a machine.Map The area of a level in a game.Master The finished product of a game/test plan.Matrices Charts and tables used in the test plan.Matrix See matrices.Microsoft The leading company of the hardware the game is to be tested on.Multiplayer - Denoting a computer game designed for or involving several players.Network A mode for playing the game over the internet with other players.Non-disclosure agreement - A contract by which one or more parties agree not to disclose confidential information that they have shared with each other as a necessary part of doing business together.Options A menu in which all settings of a game can be changed.Outline - A general description or plan showing the essential features of something but not the detail.Patching - Apatchis a piece of software designed to update acomputer programor its supporting data, to fix or improve it.Pausing When a player presses a button to bring the game to a temporary halt.PC Personal computer.Platform The system that is used to play the game (Xbox).Play-through - A more relaxed and casual focus on thegamethat is being played, as the tester plays the game through from start to finish.QA Quality assurance.Regression Testing - Regression testingis a type of softwaretestingthat seeks to uncover new software bugs, orregressions, in existing functional and non-functional areas of a system after changes such as enhancements, patches or configuration changes, have been made to them.Sanity testing - Softwaresanity testsare synonymous with smoketests. Asanityor smoke test determines whether it is possible and reasonable to continue testing. It exercises the smallest subset of application functions needed to determine whether the systems are accessible and the application logic is responsive.Save A setting a player can select to save progress of a game.SFX Sound effects.Score - The number of points, goals, runs, etc. achieved in a game or by a team or an individual.Scoring - Gain (a point, goal, run, etc.) in a competitive game.Single player - Asingle-playervideo game is a video game where input from only one playeris expected throughout the course of the gaming session.Single race A setting where the player can choose to do a single player race.Soak tests See sanity testing.Stable A build of the game that does not break easily.Test - Take measures to check the quality, performance, or reliability of (games), especially before putting it into widespread use or practice.Tested When a test has been run on a game, the tester assigned should state the game as tested.Testing See test.Testing phase The length of time testers test a game for, usually a strict number of set hours or days.Test plan - Atest planis a document detailing the objectives, target market, internal beta team, and processes for a specific betatestfor a software or hardware product. Theplantypically contains a detailed understanding of the eventual workflow.Textures The texture of graphics in the game.TV Safe Area - A term used intelevision productionto describe the areas of the television picture that can be seen on televisionscreens.UI User interface, the means by which the user and a computer system interact, in particular the use of input devices and software.Verification Testing - Verificationis the process of checking that a software system meets specifications and that it fulfils its intended purpose. It may also be referred to as software quality control.Version/V A particular form of something differing in certain respects from an earlier form or other forms of the same type of thing.Windows movie maker A program testers use to record and edit video of a game that is being played.Xbox The hardware the game is run on.

Portfolio Project 3 - Test Plan

Documents

Transcript of Portfolio Project 3 - Test Plan