© CGI Group Inc.

iT-Toolbox Testing Module Automated tests for inter-application data flows

September 2017

Overview

2

Problems and requirements 1

Solution 2

Concepts 3

Usage 4

Roadmap 5

Problems and requirements

What test-related difficulties are encountered on exchange platforms?

Flawless quality for critical data exchanges

• Major business impact in case of error (billing, logistics, …)

• Conformance / regression tests required before deployment

Complexity when testing exchanges

• Manual execution is slow, risky, costly

• Deep understanding of the platform and flows is required

Multiple and heterogenous technologies

• Transport and data formats, ESB, application-specific, …

• Dependency with an ever-changing application environment

Lack of tools dedicated to data flow test automation

3

Problems and requirements

What are the main tooling requirements?

Ensuring the absence of regression for all kinds of data flows

• During technical migrations (e.g. new exchange platform)

• During application upgrades impacting data flows

• During framework updates

• During canonical data format updates

Building a realistic test case repository

• Centralized storage for test cases (avoiding mail exchanges, local files, …)

• Keeping test cases up-to-date

Automating tests in a controlled environment

• Testing often, easily, quickly to increase quality and reactivity

• Independence from external systems when running exchange tests

• Reducing the cost of tests (configuration and execution)

4

Solution

iT-Toolbox Testing Module: a tool to automate data flow tests

Black-box testing

• Injection of input data to launch the data flow

• Collection of actual data from the output

Regression checks

• Comparison of actual and expected data

• Support for data format specifics

• Precise problem identification (line, field name, difference, …)

• Report generation

Web interface to define and execute tests with a REST API

Based on popular open-source solutions (Maven, JUnit, …) and compatible with continuous integration servers (Jenkins, …)

5

Concepts – Overview

Hierarchical test repository

• Test project: container for test campaigns (e.g. business process)

• Test campaign: container for test suites (e.g. data exchange)

• Test suite: container for test cases (e.g. flow or half-flow)

• Test case: comparison of actual/expected data (e.g. nominal, errors)

Connectors concept

• Launch flows by injecting data and collecting output

• Many transports supported (FTP, HTTP, SOAP, JMS, JDBC, etc.)

Formats concept

• Define how actual/expected data comparison is executed

• Individual fields can be named or ignored during comparison

• Many formats supported (delimited/fixed flat file, XML, JSON, etc.)

6

Concepts – Hierarchical repository

3 levels of configuration: project, campaign and test suite

• Modular configuration of connectors / formats

• Parameters can be inherited for increased reuse

• Parameters can reference properties (key/values)

7

Project Campaign

Test suite

Connectors / Formats / Properties

Connectors / Formats / Properties

Connectors / Formats / Properties

Test case Test case Test case

Concepts – Test cases

Test case execution steps:

1. Setting up the environment

2. Reading input data

3. Launching the flow

4. Collecting output data

5. Comparing actual / expected data

6. Generating difference report

7. Cleaning up the environment

Test types:

• Comparison (default)

• Error (message / exception)

• Performance (time limit)

8

Test case

Pre-action

Actual data Expected data

Format

Input data

Post-action

WS

wM

JMS

FTP

BDD Connectors

Differences

Usage – Overview

9

Definition

• Configuration of connectors and formats

• Creation of test campaigns, suites and cases

Execution

• Criteria-based definition of the test perimeter

• Launching tests

Reports

• Results analysis

• Relevant errors

• History of previous results

Usage – Definition

Web interface for test definition

• Project / campaign / suite / test case hierarchy

• Properties / connectors / formats configuration

• Test data management (upload / view / download)

10

Usage – Execution

Execution from the test repository web interface

• Selection of campaigns / suites / tests by criteria

• On-demand execution without leaving the interface

• Reports generation and history

Execution by a continuous integration server (e.g. Jenkins)

• Project downloaded and executed by CI server

• Reports generation and history

• On-demand, scheduled or triggered execution

• Notification (e.g. by mail) when a test failure occurs

Execution in standalone mode

• For use by developers / testers

• Local execution by script

• Reports generation

• Native integration with IDE such as Eclipse

11

Usage – Reports

Identification of relevant errors

• Name and path of the difference

• Detection of moved blocks between actual / expected data

• Option to ignore extra data or specific fields

12

Roadmap

Development of new features

• Productivity and user-friendliness improvements

• Fine-grained permission management in the web interface

• Easy search and browsing of test case data

• Deeper integration with Jenkins CI server

• New connectors and comparators according to user needs

Integration of a capture / mock module

• Creation of tests cases from captures

• Activation of mocks before executing tests

• webMethods-specific plugin in the works

13

Roadmap

Discover more SOA solutions by CGI

• Web site: https://it-toolbox.fr

• Email: contact@it-toolbox.fr

14

Our platforms

15

Thank you for your attention

Our commitment to you We approach every engagement with one

objective in mind: to help clients succeed