EVO22 - Test Strategy Guide

1

EVO22 Test strategy guide

Pastel Evolution

Test Strategy Guide

2


Contents Overview ........................................................................................................................................................ 3

Purpose of a Testing Strategy .................................................................................................................... 3

Procedure ................................................................................................................................................... 3

Planning for Testing ................................................................................................................................... 3

Types of Testing ......................................................................................................................................... 4

Unit Testing .................................................................................................................................................... 5

Definition ..................................................................................................................................................... 5

Phase and Work Package .......................................................................................................................... 5

Test Organization ....................................................................................................................................... 5

Test Documentation ................................................................................................................................... 5

Roles .......................................................................................................................................................... 5

System ........................................................................................................................................................ 5

Scenario Testing/Validation ........................................................................................................................... 6

Definition ..................................................................................................................................................... 6




Roles .......................................................................................................................................................... 7

System ........................................................................................................................................................ 7

Integration Testing ......................................................................................................................................... 8

Definition ..................................................................................................................................................... 8




Roles ........................................................................................................................................................ 10

System ...................................................................................................................................................... 10

System Testing ............................................................................................................................................ 11

Definition ................................................................................................................................................... 11

Phase and Work Package .......................................................................... Error! Bookmark not defined.

Roles ........................................................................................................................................................ 11

System ...................................................................................................................................................... 11

User Acceptance Testing ............................................................................................................................. 12

Definition ................................................................................................................................................... 12

Phase and Work package ........................................................................................................................ 12

Test Organization and Documentation ..................................................................................................... 12

Roles ........................................................................................................................................................ 12

System ...................................................................................................................................................... 12

Test Schedule .............................................................................................................................................. 13

Appendix ...................................................................................................................................................... 14

Unit Test ................................................................................................................................................... 14

Business Scenario Test ............................................................................................................................ 15

3


Overview This document provides an overall strategy for testing to be followed throughout the project implementation. It can be adapted to the special needs of a project.

Purpose of a Testing Strategy This testing strategy defines the requirements and goals of the Pastel Evolution configuration, determines the tools and methods used to check that the system responds correctly, determines how and when the test will be performed, and recommends how the approval process should occur.

When developing your testing strategy you have to consider test scope and test constraints based on your project plan and the implementation scope.

Testing should be considered and planned very early in an implementation project to setup additional tools (automated test tools), train the project team, and still have testing time to fit into the overall schedule.

Procedure Depending on project size, Pastel Evolution uses an iterative testing approach that builds upon itself. As you develop a test plan, you can determine which test phases are applicable for your implementation.

As you begin to configure the Pastel Evolution System, you will be unit testing your configuration settings to ensure that the system responds correctly. As you get into Final configuration, you will be testing the business processes to ensure that each process responds correctly. Additionally, there is an independent test performed at each stage to ensure integrity.

In parallel you are testing your own development (interfaces, conversions, enhancements) to ensure the correctness of the programs. After you have finished the configuration and development you will perform the Final Integration Test to ensure that the entire system including external components works correctly.

To avoid double work start testing the system with small test scenarios during Initial and Final Configuration. These short testing scenarios will be augmented to get confirmation scenarios at the end of each configuration cycle. These bigger test scenarios can be used as a basis to develop integrated test scenarios for the Final Integration Test.

Planning for Testing Different levels of testing require different levels of planning. Unit testing is at the transaction level and the testing is simple with perhaps different sets of data. At the other end of the spectrum, integration testing can be very complex. The planning and testing time for integration testing increases as the number of processes, interfaces, enhancements and configuration changes increase. If the implementation project is large and complex then having extra resources becomes necessary. A successful integration test is dependent on a thorough plan.

4


Types of Testing There are different types of testing that occur during the project, and each plays a critical role in the projects ultimate success. The testing strategy includes:

Unit testing

Scenario testing

Development testing

Integration testing

o System testing

o Technical tests

Performance tests

User acceptance testing

5


Unit Testing

Definition This is the lowest level of testing where the program or transaction is tested and evaluated for errors. Unit testing is normally the first test that is completed during the configuration effort, and is focused towards the programs inner functions, rather than towards the integration.

Testing focus:

Master data

Business processes

Examples: You can test master data by creating a Customer or a customer hierarchy. Creating a standard sales order is an example of a basic business transaction in the Sales and Distribution module.

Besides application testing unit testing is also performed during development and even during technical system testing (please refer to Development Testing and System Testing).

Phase and Work Package Unit testing will be performed as part of the baseline and final configuration work packages of the realization phase. You find several tasks for defining test cases and developing test plans during Baseline Configuration and Final Scope Configuration.

Test Organization During configuration, unit testing is done whenever it is needed. If there is a special need to organize and plan unit testing because of the project requirements create a test plan.

Test Documentation Because unit testing concentrates on testing single transactions during the ongoing configuration process there is normally no need to develop detailed test documentation. For simple transactions testing can be done straightforward during configuration.

However, some transactions are very complex involving multiple screens, functions and variations to run. Those transactions (i.e. Sales order) can be documented and tested with a BPP, maintaining the test section with test conditions and variations of the standard transaction.

Roles Consultants configure the system during Baseline configuration and perform unit testing for Baseline. During Final Scope configuration customer team members learn transactions and configuration settings, and perform some unit testing.

System Unit testing is performed in the development system, using a different testing client. Therefore the configuration settings have to be copied to the other client before testing.

Make sure that the systems you select for testing are consistent with your overall system and client landscape.

6


Scenario Testing/Validation

Definition During the configuration there is a need to test chains of transactions that flow together and which reflect important business processes and scenarios.

Testing focus:

Multiple transactions within one enterprise area

Workflow

Business processes which cross enterprise areas

Examples: (of small scenarios)

Standard sales order to a delivery to invoicing

Invoicing to account document and posting

During subsequent integration testing these small scenarios can be used to build larger end-to-end scenarios.

Phase and Work Package Scenario testing will be performed as part of the Baseline and Final Scope Configuration and Confirmation work packages of the realization phase. You find several tasks for defining test cases and developing test plans during Baseline Configuration and Final Scope Configuration.

We also recommend using test scenarios for Baseline confirmation and for Final Scope confirmation. According to Pastel Evolution at the end of Baseline Configuration as well as at the end of Final Scope configuration the most important business processes should be tested, presented and confirmed using Confirmation Scenarios. Therefore existing test scenarios can be linked and adapted to create end-to-end-scenarios that reflect the overall business flow.

Test Organization To organize and follow up the scenario testing during baseline and final configuration use the test plan.

Test Documentation To document scenario testing it is recommended using the Test Scenario template entering every single step (transaction) with input and output data. These documents can also be used for confirmation of your integrated scenarios at the end of the Baseline Configuration and for the confirmation of each cycle during Final Scope Configuration.

7


Roles The business process owner is involved in creating baseline scenarios along with the application consultant. Normally, the application consultant conducts the presentation to get the baseline confirmed. For the subsequent scenario testing and confirmation for each cycle during Final Configuration the business process owner should take on more responsibilities (i.e. defining the test scenarios).

System For scenario testing and especially for running the confirmation scenarios it is recommended to use a special test client in the development system.

8


Integration Testing

Definition Final integration testing is accomplished through the execution of predefined business flows, or scenarios, that emulate how the system will run your business. These business flows, using migrated data from the pre-existing systems, will be performed in a multifaceted computing environment comprised of Pastel Evolution, third-party software, system interfaces and various hardware and software components. It is this environment that builds the necessary level of confidence that the solution is complete and will perform in your business.

Final integration tests need to be an evolutionary process that is driven from your previous testing efforts. The test cases and scenarios that were used for Baseline and Final Configuration need to be reviewed and enhanced for the integrated test. These selected cases can be combined to represent a business process flow such as a revenue cycle or a material acquisition cycle. Problems encountered during these efforts also need to be tested under an integrated environment. Final Integration test cannot be viewed as an independent, but as a capstone effort that brings together and builds upon all previous testing efforts.

Integration testing should be allotted approximately twenty-five percent (25%) of the total time used during the Deployment Phase for configuration and testing.

Final Integration testing focuses on cross-functional integration points, as well as end-to-end business processes. A well-defined Final Integration test plan starts with the testing of the cross-functional integration points (touch points) and ends with the end-to-end testing of critical business processes identified within the Business Blueprint.

Integration test should also include testing the organizational flow, i.e. how the several organizational units are involved in one process, how they communicate, how the information and document transfer is handled.

Phase and Work Package Although there is some integration testing done during the confirmation sessions for Baseline and for Final Scope Confirmation, most of the integration testing takes place at the end of the realization phase in the Final Integration Test.

Test Organization Integration testing, as stated in the Overview, is the most complex testing in a Pastel Evolution implementation project. It is mainly because the Pastel Evolution application is an integrated system. The integration test planning involves the following.

Recommend Test Leader for test organization

Nominate a Test Leader. For smaller projects the Project Manager can fulfill this role. However, for a larger and more complex project a separate person who has already been a participant on the project would be able to focus on the Integration testing activities. It is the Test Leader who organizes the test team, and runs the workshops that accomplish the following tasks.

Establish test system

A test system, also called the Quality Assurance system (QA), should be setup by the technical team. It is necessary to have this system up and running before any integration test activity can be performed. The QA system should have been technically tested and the plan to transport configuration from the Development system should be completed.

9


Recommend test users

Build an integration test team to ensure integration thinking and development of integrated test scenarios. Nominate test users. In many cases the team members can fulfill this role. However, if the project has been lengthy or if the company is large it is a good idea to involve extended team members from the user community. This involvement by extended team members serves two purposes: it trains them and helps to get their buy-in. It is highly recommended that authentic user authorizations and logins be used for the integration testing.

Recommend one room for integration testing

Organize a separate room for the integration test. Setup personal computers, networking, printers and any other necessary hardware in this room. Establish a schedule of the necessary test users for each part of the integration test. For example, if you are testing the invoicing to cash part of a scenario then make certain that the appropriate SD and FI team members are scheduled for the room.

If space and hardware is limited then have a detailed time schedule for each part of the test. The Test Leader should coordinate and facilitate throughout the different module teams to make sure that the integration test is smooth.

Recommend cut-off criteria

It is easy to fall into a continuous loop of integration testing. However, a cut-off date and criteria should be established at the start of integration testing. The most critical testing scenarios should be used.

Scheduling and time estimates

Many of these preliminary tasks for integration testing should be completed in a timely manner. It is important to have the proper testers and systems available. During integration testing configuration changes may be required enough time for these tasks should also be scheduled.

Preparing system and test data

The critical element in an integrated test is common data. It is almost impossible to demonstrate integration without using the same data. Thus, a chart-of-accounts, business partners, materials, etc. must all be defined and properly configured as a part of the integrated test. Similarly, documents and control information (company codes, plants, organizational hierarchy, etc.) must function properly during the integrated test. A good deal of planning is required to conduct these tests (i.e. similar to planning to go into production). Successful completion of integrated tests is one of the precursors for going into production.

Test Documentation Use the Integration Test Scenario template to define your test cases.

Maintain priorities for each scenario and select only high priority test scenarios, if you cannot test everything. Test priority should be based on:

o Frequency (How often does the process occur?)

o Impact (What is the effect if the process is down?)

o Difficulty (What is the probability that problems occur?)

Concentrate on test cases which are difficult (use the 80/20 rule: 80% of test cases have few problems, and 20% have most of the problems).

Update testing dates

Assign the appropriate test user(s) for each test scenario

TIP: In the Roadmap tasks for the definition of test cases or in the accelerator list you find a list with predefined test scenarios. There are also several real-life integrated test scenarios, which show how an integration test could be performed. These test scenarios are delivered to support you in creating your own test scenarios. You can choose every scenario that meets your business, copy it and adapt it according to your needs. Create as many variations as you need to test your several business cases.

10


Roles As the knowledge transfer from the Pastel Evolution business process experts to the project team approaches an apex, responsibility for the integration test becomes that of the business process users and power users. This also instills a sense of ownership with the business process users and should make it easier to get acceptance at the end of the phase.

There is a change in the makeup of the testing teams between the configuration cycles and integration testing. As the test becomes more integrated, the test team should include team members from every enterprise area being tested. This will ensure that all processes are properly tested. Involve extended teams members to ensure you meet the companys business requirements.

System Quality Assurance System is needed for Integration testing. Master data from legacy systems should be converted onto the Q/A system. All configurations should be frozen on the Development system and transported onto the QA system.

11


System Testing

Definition System testing consists of technical tests and performance related tests. The technical tests aim to validate that the technical components of the production environment are working properly. These tests include validating the system administration procedures, failure recovery, and disaster recovery. The performance related tests include volume and stress testing of business transactions and business transaction input / output using printer and fax devices.

Technical Test

Failure Test

Disaster Recovery Test

Backup and Restore Test

System Administration Test

Printing and Fax Test

Going Live Check

Performance Test

Volume Test

Stress Test

Batch cycle test

Month/quarter/year end processing

Technical tests are rather basis oriented, whereas performance tests cover basis and application aspects. Performance tests should concentrate on stress test, i.e. stressing the system with all components involved in certain scenarios until it performs to predefined values (throughput). Volume testing is needed for some processes or transactions, if the throughput of one process/transaction has to meet predefined requirements (i.e. creating large amounts of sales orders, daily interfaces, etc.)

Roles Technical Team Lead (along with application team members).

System Testing can be prepared in the Quality Assurance system, but must run on the Production system.

12


User Acceptance Testing

Definition Normally there is no special need for a User Acceptance Test, because the system should have been accepted by the confirmation of the Final Integration Testing. If you need to perform a separate User Acceptance Test, you can choose some of the Integration Test Scenarios.

Phase and Work package Final Preparation (optional, no Roadmap work package).

Test Organization and Documentation Use some of your existing Integration Test Scenarios to perform and document the test.

Roles Power User, End User.

System Quality Assurance system and Production system

13


Test Schedule

Initial Configuration Final Configuration Integration Test

IT 1 IT 2

Development of Conversions, Interfaces, ...

Unit Test: BPP with test conditions

Scenario Test: Test Scenario template

Unit Test Development: Functional specification test section

Integrated test scenario

IT scenario with develop.

14


Appendix The purpose of this appendix is to provide a template for the development of a plan for testing and validation of baseline and final configuration.

Unit Test

Description

Test Tool

Test System

Roles and Responsibilities

Test Data

Test Document

Test Cases

Test Schedule and Milestones

Test Deliverables

Test Execution and Results

Test Status and Tracking

15


Business Scenario Test

Description

Test Tool

Test System

Roles and Responsibilities

Test Data

Test Document

Test Cases

Test Schedule and Milestones

Test Deliverables

Test Execution and Results

Test Status and Tracking

EVO22 - Test Strategy Guide

Documents

Transcript of EVO22 - Test Strategy Guide