Presenter:Miguel Garzon Torres

CrUise Lab - SITE

SQL Coverage Measurement for Testing Database

Applications

María José Suárez-CabalUniversity of Oviedo

Javier TuyaUniversity of Oviedo

2

Glossary

• DBMS:DBMS: Database Management System

• SQL:SQL: Structured Query Language

• UML:UML: Unified Modeling Language

3

Context

• A software application that has a Database

• SQL Queries are embedded in the application code– Queries are part of the application’s business

logic

4

Goal

• Goal: We want to achieve the highest possible value of coverage of queries that access to the database.

• In other words, we want to test the database and obtain the highest SQL Coverage possible.

• HOW ?– Database test data is manipulated in order to

get a better coverage

5

Motivation

• Testing is a very expensive process in software development – 30% of resources committed to this

• Most applications use a database for accessing the contents that they offer.

• We need to test the database – Because SQL queries are embedded into the code,

we need to test in a similar way we test the rest of the code

– Tests should cover all query situations and obtain maximum possible coverage

6

Introduction

• The main aims of the research are:– Define a measurement of coverage of SQL

Select queries in databases loaded with test data

– Present an algorithm that automates the calculation of coverage

– Guide the expert in the manipulation of test data to obtain better results in terms of coverage

7

Related Work –Research

• Microsoft Research: Valid SQL statements were generated and run on several systems that contain identical databases– Goal: Evaluate DBMS and compare output

• AGENDA: Test data is generated, SQL queries are executed and the state of the database is verified.

• Other studies did not consider database schema and the database integrity constraints

8

Related Work - Commercial

• Generators of database instances: TestByte, TestBase, DataTect generate random information depending on fields

• Users must know the database structure in order to use these tools

• SQLUnit is used for testing of stored procedures and SQL queries (XML)

9

Software testing with DBs

• Testing is very difficult:– Selection of the initial instance of the test

database is one of the most important steps– Combinations between tuples must be taken

into account to cover all the SQL query situations

– Test data should be useful for the greatest possible number of queries

10

Simple Example

sup_codesup_code sup_namesup_name

001 sup1

002 sup2

ord_numord_num sup_codesup_code

1 001

2 001

3 002

1 2

ord_numord_num sup_namesup_name

1 sup1

2 sup1

2 sup2

SELECT sup_name, ord_numFROM Supplier s LEFT JOIN Order o

ON (s.sup_code=o.sup_code)

11

Detecting faults in SELECT queries

• Problems detected:– There may be suppliers with no orders, but

this situation is not represented by the test data (all suppliers have at least one order)

– Impossible to know whether the query always returns the information required

– So, an adequate SELECT coverage is not obtained with these test data.

12

Detecting faults in SELECT queries (cont’d)

sup_codesup_code sup_namesup_name

001 sup1

002 sup2

003 sup3

ord_numord_num sup_codesup_code

1 001

2 001

3 002

ord_numord_num sup_namesup_name

1 sup1

2 sup1

2 sup2

sup3

SELECT sup_name, ord_numFROM Supplier s LEFT JOIN Order o

ON (s.sup_code=o.sup_code)

13

Coverage of SELECT Queries

1. SELECT Query:

14

Coverage of SELECT Queries

2. Coverage tree and evaluation of conditions:

• Evaluate the conditions of SELECT queries that are in the FROM clause, when they include JOIN, and in the WHERE clause

• Null values of fields will be verified as the conditions are evaluated

15

Coverage of SELECT Queries –Coverage Tree

• Each level represents a condition of the query:– begins with conditions of JOIN clause (if it

exists)– then those of WHERE clause – In same order as they appear in the query

• Each node of the tree will store:– Whether the condition is true for values of the

fields;

16

Coverage of SELECT Queries –Coverage Tree (cont’d)

• During the evaluation of a condition:– Each value in the first field must be compared

with each one in the second field– Each value in the second field with each one

in the first

17

Coverage of SELECT Queries –Coverage Tree (cont’d)

• The evaluation of a condition:The grammar:

A condition will be true if it is verified for a pair of values from the fields to compare.

A condition will be false from left to right, if none of the values from the second field verifies the condition with a value from the first field.

A condition will be false from right to left, if none of the values from the first field verifies the condition with a value from the second field.

18

Coverage of SELECT Queries –Coverage Tree (cont’d)

• The evaluation of a condition:

COVERAGE TREE:

A condition will have null values when a value from the first field is null.

A condition will have null values when a value from the second field is null.

A condition will have null values when a value from the first and second fields are null.

19

Coverage of SELECT Queries –Coverage Tree (cont’d)

• Coverage Tree

20

Evaluation of the coverage tree

• The evaluation is made by crossing over the tuples of the tables participating in the conditions at each level of the tree.– Finishes when 100% coverage or no more values

• Two different coverage measures are established:– Theoretical coverage: Takes into account every

possible situation at every node– Schema coverage: Takes into account the database

schema constraints

21

Theoretical Coverage

• It is not usually possible to reach 100% theoretical coverage, because of forbidden null values or referential integrity constraints.

• Solution: Schema coverage

22

Schema Coverage

• Due to referential integrity, an order always had a supplier. Fr, Nb, and Nr will never be verified (X).

23

Tool Demonstration

Generation of Full Predicate Logic Coverage RulesGeneration of Full Predicate Logic Coverage Rules

11

22

33

Enter Database Schema

Enter SQL Query

Complete the test data from Rules generated

24

Tool Demonstration – Other Tool

Case Study

• The company’s department is responsible for managing the lamination rolls used in the rolling mills for the manufacture of steel sheets.

• Rolls are arranged in boxes. A mill may consist of one or more boxes.

• Company manages thousands of rolls, as well of dozens of boxes and mills.

Case Study - The problem

• All information about rolls, mills and boxes are maintained in a database.

• The database used for testing has about a thousand rolls and twenty boxes and mills.

Case Study - Queries

• Several SELECT queries that contain distinct tables of the system have been analyzed with the tool.

Case Study – Data model

UML ModelUML Model

Database Database TablesTables

Database Database TablesTables

Case Study – Coverage Tree

Case Study – Measuring coverage

• After running the algorithm, we obtain:

– Theoretical coverage = 19.23%

• Next steps:– Simplification of tuples based on

constraints– Completing test data to complete our testing

31

Future Work

• Including parameters in SQL QueryIncluding parameters in SQL Query

• Support for stored proceduresSupport for stored procedures

• More case studies demonstrating the More case studies demonstrating the conceptsconcepts

32

Questions?

Miguel Garzón - mgarz042@uottawa.ca