Property Based Testing

Mårten Rånge

Atomize - Automatic revenue management

A simple idea...

Create a test for a

property of our code

Let the computer

generate test data

If the property held for

all data the test passed

Property Based Testing done properly...

▸Improves productivity

▸Improves test quality

▸Finds more bugs during development

Map<'K, 'V>

§1 – The map is immutable

§2 – Lookup after set shall succeed

§3 – Lookup after unset shall fail

Red Tree

3

1

~ ~

5

~ 7

~ ~

§4 – Parent key is greater than left child,smaller than right child

§5 – No Red Node has a Red child

§6 – Every path from the Root to a Leaf containsthe same number of nodes

§7 – §5 + §6 Tree depth at most 2[log2 (n + 1)]

3

1

~ ~

5

~ 7

~ ~

type Color =| Red| Black

type Tree<'K, 'V> =| Leaf // Left child Right child| Node of Color*Tree<'K, 'V>*'K*'V*Tree<'K, 'V>

val set: 'K -> 'V -> Tree<'K, 'V> -> Tree<'K, 'V>

val lookup: 'K -> Tree<'K, 'V> -> 'V option

Demo

Common patterns

Oracles

static member ``test chunkBySize`` (sz : int) (vs : int []) =let sz = wrap sz - 1 vs.Lengthlet e = vs |> chunkBySize szlet a = vs |> Stream.ofArray

|> Stream.chunkBySize sz |> Stream.toArray

e = a

static member ``test collect`` (vs : int [] []) =let e = vs |> Array.collect idlet a = vs |> Stream.ofArray

|> Stream.collect Stream.ofArray |> Stream.toArray

e = a

Actions

type HeapActions =| Push of int*int64| Pop

static member ``test`` (a : HeapActions []) =...

Test Models

type Json =| JsonNull| JsonBoolean of bool| JsonNumber of float| JsonString of string| JsonArray of Json []| JsonObject of (string*Json) []

type Ws = Whitespace []* Whitespace []

type WsJson =| WsJsonNull Ws| WsJsonBoolean of Ws*bool| WsJsonNumber of Ws*float| WsJsonString of Ws*string| WsJsonArray of Ws*WsJson []| WsJsonObject of Ws*((Ws*string)*Json) []

Randomized testing?

Pseudo randomized testing

The seed is traced

Reproducible

Specify the seed

Predictable

Property Based Testing Frameworks

▸QuickCheck (Haskell)

▸FsCheck (.NET)

▸ScalaCheck (Scala)

▸JUnit-QuickCheck (Java)

Questions?

marten@atomize.com