MapReduce for ParallelTrace Validation of LTL Properties
Benjamin Barre, Mathieu Klein, Maxime Soucy-Boivin,Pierre-Antoine Ollivier and Sylvain HallΓ©
UniversitΓ© du QuΓ©bec Γ ChicoutimiCANADA
CRSNGNSERC
Fonds de rechercheNature ettechnologies
System
System
System
Instrumentation
System
Instrumentation
System
Instrumentation
Trace
System
Instrumentation
Trace
Events
System
Instrumentation
Trace
Events
System
Instrumentation
Trace
Events
Tracevalidation
Iterator<T>
Iterator<T>
hasNext
next
Iterator<T>
hasNext
next
A call to next must be precededby a call to hasNext
B
A
B
A
No CartCreate request can occurbefore a LoginResponse message
Login
Login
Three successive login attemptsshould trigger an alarm
Receive order
Receive orderReady?
Receive orderReady? Yes
Receive orderReady? Yes
File order
No Ship
Receive orderReady? Yes
File order
No Ship
A received order must eventuallybe shipped
A
0 1 2 3 4 . . .
a a b c b
β
A trace m is a mapping from β tothe set of events :
ALet be a set of event symbols.
A
Groundterms
βΒ¬β§βΒ¬β§
Booleanconnectives
Temporaloperators
XGFU
nextgloballyeventuallyuntil
+ +
= Linear Temporal Logic
A
0 1 2 3 4 . . .
a a b c b
β
Ξ¦Let be the set of all possible LTL formulas.
The function β : Ξ¦ β 2 labels each state witha set of LTL formulas
β
aβ§b
aβ§b
G (aβb)
bβ¨c
bβ¨c
aβ§b
G (aβb)
β(aβ§b) = {0,1,4,...}Example:
β
A
0 1 2 3 4 . . .
a a b c b
β
Ξ¦Let be the set of all possible LTL formulas.
The function β : Ξ¦ β 2 labels each state witha set of LTL formulas
β
i β β(Οβ¨Ο) β i β β(Ο) or i β β(Ο)i β β(Οβ§Ο) β i β β(Ο) and i β β(Ο)i β β(Β¬Ο) β i β β(Ο)
i β β(G Ο) β j β β(Ο) for all j β₯ ii β β(X Ο) β i+1 β β(Ο)
i β β(F Ο) β j β β(Ο) for some j β₯ ii β β(Ο U Ο) β j β β(Ο) for some j β₯ i and
k β β(Ο) for all j β₯ k β₯ i
i β β(a) β m(i) = a
i β β(Ο) exactly when the tracem(i), m(i+1), ... satisfies Ο
Theorem
ΟΟ Ο
0 1 2 3 4 . . .
ΟΟ Ο
i β β(Ο) exactly when the tracem(i), m(i+1), ... satisfies Ο
Theorem
ΟΟ Ο
0 1 2 3 4 . . .
ΟΟ Ο
0 β β(Ο) β m β§ Ο
Therefore...
A call to next must be followed by a callto hasNext
No CartCreate request can occurbefore a LoginResponse message
A received order must eventuallybe shipped
Three successive login attempts shouldtrigger an alarm
A call to next must be followed by a callto hasNext
No CartCreate request can occurbefore a LoginResponse message
A received order must eventuallybe shipped
Three successive login attempts shouldtrigger an alarm
G (next β X hasNext)
A call to next must be followed by a callto hasNext
No CartCreate request can occurbefore a LoginResponse message
A received order must eventuallybe shipped
Three successive login attempts shouldtrigger an alarm
G (next β X hasNext)
Β¬ CartCreate U hasNext
A call to next must be followed by a callto hasNext
No CartCreate request can occurbefore a LoginResponse message
A received order must eventuallybe shipped
Three successive login attempts shouldtrigger an alarm
G (next β X hasNext)
Β¬ CartCreate U hasNext
G (receive β F ship)
A call to next must be followed by a callto hasNext
No CartCreate request can occurbefore a LoginResponse message
A received order must eventuallybe shipped
Three successive login attempts shouldtrigger an alarm
G (next β X hasNext)
Β¬ CartCreate U hasNext
G (receive β F ship)
G Β¬(fail β§ (X (fail β§ X fail)))
Iterator<T> Java MOP
21 3 4 5
The trace mustbe read linearly
The algorithm works on asingle process / core / sitex1
οΏ½
οΏ½
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
1970 1980 1990 2000 2010
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
1970 1980 1990 2000 2010
Transistors (x1000)
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
1970 1980 1990 2000 2010
Transistors (x1000)
CPU Speed
(MHz)
fβPageRank
a 1
KeyValue
Tuple (baaah){
Data source
Data source
IIInput reader
Data source
IIInput reader
. . .2 7a z
2a. . . 2a
2a. . . 2a M
Mapper
2a. . . 2a M
Mapper
2a. . .6w
a 23 g
a
3b3 b
2a. . . 2a M
Mapper
2a. . .6w
3 aa 2
3a3a
b 9 3 ae 83a
bb
a
ab
Shuffling
3 ae 8
ba
. . .
b
. . .
b
aa
bd
a
a
a 2
3a
b
b 9
aa2a 2 3a
b9b
aa2a 2 3a
b9b
Ra
Rb
Reducer
aa2a 2 3a
b9b
Ra
Rb
Reducer
. . .z 8 x 2
e 7 i 0
a b a a b a
a b a a b a
ab
ba
a a
a b a a b a
ab
ba
a a I
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
γb,1γ
γa,1γI
Iγa,1γ
γb,1γ
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
γb,1γ
γa,1γI
Iγa,1γ
γb,1γ
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
γb,1γ
γa,1γI
Iγa,1γ
γb,1γ
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
γb,1γ
γa,1γI
Iγa,1γ
γb,1γ
Ra
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
γb,1γ
γa,1γI
Iγa,1γ
γb,1γ
Ra γa,4γ
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
γb,1γ
γa,1γI
Iγa,1γ
γb,1γ
Ra γa,4γ
γb,2γRb
a b a a b a
ab
ba
a a Iγa,1γ
γa,1γ
γb,1γ
γa,1γI
Iγa,1γ
γb,1γ
Ra γa,4γ
γb,2γRb
GG β§β§
Subformula
Superformula
Formula
Subformula Subformula
Superformula
1
0
2
3
β§
a c b
Β¬ F
G
1
0
2
4Height
β3
1
0
2
3
β§
a c b
Β¬ F
G
1
0
2
4Height
β3
Β¬c has height 1G ((a β§Β¬c) β F b) has height 4
i β β(Οβ¨Ο) β i β β(Ο) or i β β(Ο)i β β(Οβ§Ο) β i β β(Ο) and i β β(Ο)i β β(Β¬Ο) β i β β(Ο)
i β β(G Ο) β j β β(Ο) for all j β₯ ii β β(X Ο) β i+1 β β(Ο)
i β β(F Ο) β j β β(Ο) for some j β₯ ii β β(Ο U Ο) β j β β(Ο) for some j β₯ i and
k β β(Ο) for all j β₯ k β₯ i
i β β(a) β m(i) = a
The labelling of a formula depends onlyon labellings of formulas of strictly lower height
i β β(Οβ¨Ο) β i β β(Ο) or i β β(Ο)i β β(Οβ§Ο) β i β β(Ο) and i β β(Ο)i β β(Β¬Ο) β i β β(Ο)
i β β(G Ο) β j β β(Ο) for all j β₯ ii β β(X Ο) β i+1 β β(Ο)
i β β(F Ο) β j β β(Ο) for some j β₯ ii β β(Ο U Ο) β j β β(Ο) for some j β₯ i and
k β β(Ο) for all j β₯ k β₯ i
i β β(a) β m(i) = a
The labelling of a formula depends onlyon labellings of formulas of strictly lower height
All labellings of formulas of same height are independent
β
i β β(Οβ¨Ο) β i β β(Ο) or i β β(Ο)i β β(Οβ§Ο) β i β β(Ο) and i β β(Ο)i β β(Β¬Ο) β i β β(Ο)
i β β(G Ο) β j β β(Ο) for all j β₯ ii β β(X Ο) β i+1 β β(Ο)
i β β(F Ο) β j β β(Ο) for some j β₯ ii β β(Ο U Ο) β j β β(Ο) for some j β₯ i and
k β β(Ο) for all j β₯ k β₯ i
i β β(a) β m(i) = a
The labelling of a formula depends onlyon labellings of formulas of strictly lower height
All labellings of formulas of same height are independent
β
β They can be computed in parallel
i β β(Οβ¨Ο) β i β β(Ο) or i β β(Ο)i β β(Οβ§Ο) β i β β(Ο) and i β β(Ο)i β β(Β¬Ο) β i β β(Ο)
i β β(G Ο) β j β β(Ο) for all j β₯ ii β β(X Ο) β i+1 β β(Ο)
i β β(F Ο) β j β β(Ο) for some j β₯ ii β β(Ο U Ο) β j β β(Ο) for some j β₯ i and
k β β(Ο) for all j β₯ k β₯ i
i β β(a) β m(i) = a
M
Input: tuples γΟ,(n,i)γ
M
Input: tuples γΟ,(n,i)γ
β n β β(Ο), and the last cycle has evaluatedlabellings for formulas of height i β
M
Input: tuples γΟ,(n,i)γ
β n β β(Ο), and the last cycle has evaluatedlabellings for formulas of height i β
M βLiftβ β(Ο) to superformulas of Ο
Input: tuples γΟ,(n,i)γ
β n β β(Ο), and the last cycle has evaluatedlabellings for formulas of height i β
M βLiftβ β(Ο) to superformulas of Ο
Output: tuples γΟ,(Ο,n,i)γ
β n β β(Ο), the last cycle has evaluatedlabellings for formulas of height i, andΟ is a subformula of Ο β
Input: tuples γΟ,(n,i)γ
β n β β(Ο), and the last cycle has evaluatedlabellings for formulas of height i β
M βLiftβ β(Ο) to superformulas of Ο
Output: tuples γΟ,(Ο,n,i)γ
RΟ
RΟ
Input:γΟ,(Ο,n,i)γ
RΟ
Input:γΟ,(Ο,n,i)γ
β n β β(Ο), the last cyclehas evaluated labellings forformulas of height i, andΟ is a subformula of Ο β
RΟ
Input:γΟ,(Ο,n,i)γ
Compute β(Ο)
β n β β(Ο), the last cyclehas evaluated labellings forformulas of height i, andΟ is a subformula of Ο β
RΟ
Input:γΟ,(Ο,n,i)γ
Compute β(Ο)
β n β β(Ο), the last cyclehas evaluated labellings forformulas of height i, andΟ is a subformula of Ο β
Output:γΟ,(n,i+1)γ
RΟ
Input:γΟ,(Ο,n,i)γ
Compute β(Ο)
β n β β(Ο), the last cyclehas evaluated labellings forformulas of height i, andΟ is a subformula of Ο β
Output:γΟ,(n,i+1)γ
β n β β(Ο), and the lastcycle has evaluatedlabellings for formulas of height i+1
I
I
Input: events (a,n)
I
Input: events (a,n)
Output: tuples γΟ,(a,n,0)γ
. . .
β n β β(a), the last cycle has evaluatedlabellings for formulas of height 0, anda is a subformula of Ο β
W
Input: γΟ,(n,i)γ
W
Input: γΟ,(n,i)γ
W
Output:
True if γΟ,(0,i)γis read
False otherwise
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
InputReaders generate the first tuples fromthe trace chunks
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
The tuples are shuffled to reducers that compute thelabelling β for formulas of height 1
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
Mappers copy the labellings into tuples marked bysuperformulas of height 2
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
Each reducer computes the labelling of a formula ofheight 2 from the labelling of its subformulas
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
Mappers copy the labellings into tuples marked bysuperformulas of height 3
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
Each reducer computes the labelling of a formula ofheight 3 from the labelling of its subformulas
1
2
3
. . .II
RR
R
RR
R
R
RW
. . .
An output writer collects the resulting tuples, andoutputs βtrueβ if it encounters a tuple for state 0
β¨ G (Β¬a β F b)?
a a b c b a
β¨ G (Β¬a β F b)?
a a b c b a
(a,0)
(a,1)
(a,5)
(b,2)
(c,3)
(b,4)
0HEIGHT
β¨ G (Β¬a β F b)?
a a b c b a
(a,0)
(a,1)
(a,5)
(b,2)
(c,3)
(b,4)
0HEIGHT
I
I
I
β¨ G (Β¬a β F b)?
a a b c b a
(a,0)
(a,1)
(a,5)
(b,2)
(c,3)
(b,4)
0HEIGHT
I
I
I
γΒ¬a,(a,0)γ
γΒ¬a,(a,1)γ
γΒ¬a,(a,5)γ
γF b,(b,4)γ
γF b,(b,2)γ
1HEIGHT
β¨ G (Β¬a β F b)?
a a b c b a
(a,0)
(a,1)
(a,5)
(b,2)
(c,3)
(b,4)
0HEIGHT
I
I
I
γΒ¬a,(a,0)γ
γΒ¬a,(a,1)γ
γΒ¬a,(a,5)γ
γF b,(b,4)γ
γF b,(b,2)γ
1HEIGHT
R¬a
RF b
β¨ G (Β¬a β F b)?
a a b c b a
(a,0)
(a,1)
(a,5)
(b,2)
(c,3)
(b,4)
0HEIGHT
I
I
I
γΒ¬a,(a,0)γ
γΒ¬a,(a,1)γ
γΒ¬a,(a,5)γ
γF b,(b,4)γ
γF b,(b,2)γ
1HEIGHT
R¬a
RF b
γΒ¬a,2γγΒ¬a,3γγΒ¬a,4γ
γF b,0γ
γF b,1γ
γF b,2γ
γF b,3γ
γF b,4γ
β¨ G (Β¬a β F b)?
a a b c b a
γΒ¬a,2γγΒ¬a,3γγΒ¬a,4γ
γF b,0γγF b,1γγF b,2γ
γF b,3γγF b,4γ
M
M
M
2HEIGHT
β¨ G (Β¬a β F b)?
a a b c b a
γΒ¬a,2γγΒ¬a,3γγΒ¬a,4γ
γF b,0γγF b,1γγF b,2γ
γF b,3γγF b,4γ
M
M
M
2HEIGHT
γΒ¬a β F b,(Β¬a,2)γγΒ¬a β F b,(Β¬a,3)γγΒ¬a β F b,(Β¬a,4)γ
γΒ¬a β F b,(F b,0)γγΒ¬a β F b,(F b,1)γγΒ¬a β F b,(F b,2)γ
γΒ¬a β F b,(F b,3)γ
γΒ¬a β F b,(F b,4)γ
β¨ G (Β¬a β F b)?
a a b c b a
γΒ¬a,2γγΒ¬a,3γγΒ¬a,4γ
γF b,0γγF b,1γγF b,2γ
γF b,3γγF b,4γ
M
M
M
2HEIGHT
γΒ¬a β F b,(Β¬a,2)γγΒ¬a β F b,(Β¬a,3)γγΒ¬a β F b,(Β¬a,4)γ
γΒ¬a β F b,(F b,0)γγΒ¬a β F b,(F b,1)γγΒ¬a β F b,(F b,2)γ
γΒ¬a β F b,(F b,3)γ
γΒ¬a β F b,(F b,4)γ
RΒ¬a β
F b
β¨ G (Β¬a β F b)?
a a b c b a
γΒ¬a,2γγΒ¬a,3γγΒ¬a,4γ
γF b,0γγF b,1γγF b,2γ
γF b,3γγF b,4γ
M
M
M
2HEIGHT
γΒ¬a β F b,(Β¬a,2)γγΒ¬a β F b,(Β¬a,3)γγΒ¬a β F b,(Β¬a,4)γ
γΒ¬a β F b,(F b,0)γγΒ¬a β F b,(F b,1)γγΒ¬a β F b,(F b,2)γ
γΒ¬a β F b,(F b,3)γ
γΒ¬a β F b,(F b,4)γ
RΒ¬a β
F b
γΒ¬a β F b,2γγΒ¬a β F b,1γγΒ¬a β F b,0γ
γΒ¬a β F b,3γγΒ¬a β F b,4γγΒ¬a β F b,5γ
β¨ G (Β¬a β F b)?
a a b c b a
M
M
M
γΒ¬a β F b,2γ
γΒ¬a β F b,1γγΒ¬a β F b,0γ
γΒ¬a β F b,3γ
γΒ¬a β F b,4γγΒ¬a β F b,5γ
3HEIGHT
β¨ G (Β¬a β F b)?
a a b c b a
M
M
M
γΒ¬a β F b,2γ
γΒ¬a β F b,1γγΒ¬a β F b,0γ
γΒ¬a β F b,3γ
γΒ¬a β F b,4γγΒ¬a β F b,5γ
3HEIGHT
γG (Β¬a β F b), (Β¬a β F b,0)γ
γG (Β¬a β F b), (Β¬a β F b,1)γ
γG (Β¬a β F b), (Β¬a β F b,2)γ
γG (Β¬a β F b), (Β¬a β F b,3)γ
γG (Β¬a β F b), (Β¬a β F b,4)γ
γG (Β¬a β F b), (Β¬a β F b,5)γ
β¨ G (Β¬a β F b)?
a a b c b a
M
M
M
γΒ¬a β F b,2γ
γΒ¬a β F b,1γγΒ¬a β F b,0γ
γΒ¬a β F b,3γ
γΒ¬a β F b,4γγΒ¬a β F b,5γ
3HEIGHT
γG (Β¬a β F b), (Β¬a β F b,0)γ
γG (Β¬a β F b), (Β¬a β F b,1)γ
γG (Β¬a β F b), (Β¬a β F b,2)γ
γG (Β¬a β F b), (Β¬a β F b,3)γ
γG (Β¬a β F b), (Β¬a β F b,4)γ
γG (Β¬a β F b), (Β¬a β F b,5)γ
RG (Β¬aβ F b)
β¨ G (Β¬a β F b)?
a a b c b a
M
M
M
γΒ¬a β F b,2γ
γΒ¬a β F b,1γγΒ¬a β F b,0γ
γΒ¬a β F b,3γ
γΒ¬a β F b,4γγΒ¬a β F b,5γ
3HEIGHT
γG (Β¬a β F b), (Β¬a β F b,0)γ
γG (Β¬a β F b), (Β¬a β F b,1)γ
γG (Β¬a β F b), (Β¬a β F b,2)γ
γG (Β¬a β F b), (Β¬a β F b,3)γ
γG (Β¬a β F b), (Β¬a β F b,4)γ
γG (Β¬a β F b), (Β¬a β F b,5)γ
RG (Β¬aβ F b)
γG (Β¬a β F b),0γγG (Β¬a β F b),1γγG (Β¬a β F b),2γγG (Β¬a β F b),3γγG (Β¬a β F b),4γγG (Β¬a β F b),5γ
β¨ G (Β¬a β F b)?
a a b c b a
WγG (Β¬a β F b),0γγG (Β¬a β F b),1γγG (Β¬a β F b),2γγG (Β¬a β F b),3γγG (Β¬a β F b),4γγG (Β¬a β F b),5γ
4HEIGHT
β¨ G (Β¬a β F b)?
a a b c b a
WγG (Β¬a β F b),0γγG (Β¬a β F b),1γγG (Β¬a β F b),2γγG (Β¬a β F b),3γγG (Β¬a β F b),4γγG (Β¬a β F b),5γ
4HEIGHT
True
The trace can be stored inseparate (and non-contiguous)chunks
Mappers and reducers of agiven height can operate
in parallel
οΏ½ (a,0) (b,2)(a,1) (c,3)
(a,5) (b,4)
οΏ½R
RR
MM
M
Tests on 500 randomly-generated traces
From 1 to 100,000 events
Each event contains 10 parametersnamed pβ to pβ with 10 possible values
G pβ β 0
G (pβ = 0 β X pβ = 0)
βx β [0,9] : G (pβ = x β X pβ = x)
βm β [0,9] : βx β [0,9] : G (p = x β X X p β x)m m
1
2
3
4
Validation of 4 LTL formulas:
PropertyTuples
Time/eventSequential ratio
Inferred time
55 k19 ΞΌs100%19 ΞΌs
120 k23 ΞΌs92%21 ΞΌs
600 k75 ΞΌs92%14 ΞΌs
5 M985 ΞΌs
3%30 ΞΌs
1 2 3 4
MQuestions?
Top Related