A Case for "Piggyback" Runtime Monitoring
147
THE FOLLOWING HAS BEEN APPROVED FOR TALK OPEN-MINDED AUDIENCES THE FOLLOWING HAS BEEN APPROVED FOR TALK OPEN-MINDED AUDIENCES R R S SCIENTIFIC RESEARCH CONTENT CONTROVERSIAL IDEAS, CONJECTURES BACKED BY PRELIMINARY EXPERIMENTAL DATA S SCIENTIFIC RESEARCH CONTENT CONTROVERSIAL IDEAS, CONJECTURES BACKED BY PRELIMINARY EXPERIMENTAL DATA
-
Upload
sylvain-halle -
Category
Documents
-
view
589 -
download
0
description
A runtime monitor enforcing a constraint on sequences of method calls on an object must keep track of the state of the sequence by updating an appropriate state machine. The present paper stems from the observation that an object's member fields must already contain an encoding of that state machine, and that a monitor essentially duplicates operations that the object performs internally. Rather than maintain a state machine in parallel, the paper puts forward the concept of "piggyback" runtime monitoring, where the monitor relies as much as possible on the object's own state variables to perform its task. Experiments on real-world benchmarks show that this approach greatly simplifies the monitoring process and drastically reduces the incurred runtime overhead compared to classical solutions.
Transcript of A Case for "Piggyback" Runtime Monitoring
THE FOLLOWING HAS BEEN APPROVED FORTALKOPEN-MINDED AUDIENCES
THE FOLLOWING HAS BEEN APPROVED FORTALKOPEN-MINDED AUDIENCES
RR
S SCIENTIFIC RESEARCH CONTENT
CONTROVERSIAL IDEAS, CONJECTURESBACKED BY PRELIMINARY EXPERIMENTAL DATA
S SCIENTIFIC RESEARCH CONTENT
CONTROVERSIAL IDEAS, CONJECTURESBACKED BY PRELIMINARY EXPERIMENTAL DATA
PrologueWhat Plato taught us
A Case for "Piggyback"Runtime Monitoring
Raphaël Tremblay-Lessard and Sylvain Hallé
Laboratoire d'informatique formelleUniversité du Québec à Chicoutimi, Canada
Fonds de recherchesur la natureet les technologies
CRSNGNSERC
System
System
System
Instrumentation
System
Instrumentation
System
Instrumentation
Trace
System
Instrumentation
Trace
Events
System
Instrumentation
Trace
Events
System
Instrumentation
Trace
Events
Tracevalidation
System
Instrumentation
System
Runtime monitoringInstrumentation
System
Runtime monitoringInstrumentation
System
Runtime monitoring
Overhead
Instrumentation
We consider a set A of method calls on a
single instance of some class C
We consider a set A of method calls on a
single instance of some class C
C distinguishes between “valid” and
“invalid” sequences of method calls in A
We consider a set A of method calls on a
single instance of some class C
C distinguishes between “valid” and
“invalid” sequences of method calls in A
C is deterministic: for every sequence
m ∈ A*, m is either always valid or
always invalid
→∧( )
→∧( )
→∨( )∅ X
My car leaves
Mailman delivers mail
Neighbor picks mail
My car returns
*
*
*
Error
Iterator<T>
Iterator<T>
hasNext
next
Iterator<T>
hasNext
next
hasNext
next
hasNext
Open for read
CloseRead
WriteOpen for write
Close
class MyHouse {
boolean isHome = false; boolean hasMail = false;
public void carLeaves() { isHome = false; } public void carReturns() { isHome = true; } public void mailman() { hasMail = true; } public void pickMail() { if (hasMail && !isHome) hasMail = false; else Do Something Bad; }}
There exists a minimal DFA M = 〈Q,q₀,δ〉such that ℒ(M) = all valid sequences of
method calls in A
Let q = q₀Wait for method call m ∈ ALet q = δ(q, m)
If q = then raise error
Else goto 2
1
2
3
4
5
m1.carLeaves();
m2.mailman();
m1.pickMail();
MyProgram:
. . .
m1.carLeaves();
m2.mailman();
m1.pickMail();
MyProgram:
. . .
aspect Monitor (MyHouse m) { int state = 0;
pointcut before carLeaves() { if (state == 0) state = 1; } pointcut before carReturns() { if (state == 1) state = 0; } pointcut before mailman() { if (state == 1) state = 2; } pointcut before pickMail() { if (state == 2) state = 1; }
m1.carLeaves();
m2.mailman();
m1.pickMail();
MyProgram:
. . .
aspect Monitor (MyHouse m) { int state = 0;
pointcut before carLeaves() { if (state == 0) state = 1; } pointcut before carReturns() { if (state == 1) state = 0; } pointcut before mailman() { if (state == 1) state = 2; } pointcut before pickMail() { if (state == 2) state = 1; }
Non-intrusive...
...but "memoryful" (must persist q)
Lots of events to watch
OVERHEAD
Neighbor about to pick mail
?
Is there a car?
?
Is there mailin the box?
Key point
Key point
Don't observemethod calls...
Key point
Don't observemethod calls...
...query the object'sstate directly !
??
class MyHouse {
boolean isHome = false; boolean hasMail = false;
public void carLeaves() { isHome = false; } public void carReturns() { isHome = true; } public void mailman() { hasMail = true; } public void pickMail() { if (hasMail && !isHome) hasMail = false; else Do Something Bad; }}
aspect Monitor (MyHouse m) { int state = 0;
pointcut before carLeaves() { if (state == 0) state = 1; } pointcut before carReturns() { if (state == 1) state = 0; } pointcut before mailman() { if (state == 1) state = 2; } pointcut before pickMail() { if (state == 2) state = 1; }
class MyHouse {
boolean isHome = false; boolean hasMail = false;
public void carLeaves() { isHome = false; } public void carReturns() { isHome = true; } public void mailman() { hasMail = true; } public void pickMail() { if (hasMail && !isHome) hasMail = false; else Do Something Bad; }}
aspect Monitor (MyHouse m) {
pointcut before pickMail() { if (m.isHome || !m.hasMail) // Error }
class MyHouse {
boolean isHome = false; boolean hasMail = false;
public void carLeaves() { isHome = false; } public void carReturns() { isHome = true; } public void mailman() { hasMail = true; } public void pickMail() { if (hasMail && !isHome) hasMail = false; else Do Something Bad; }}
aspect Monitor (MyHouse m) {
pointcut before pickMail() { if (m.isHome || !m.hasMail) // Error }
No state to persist
Only one method to
watch
Same effect !
Key point (again)
The first monitor observes method calls to
deduce the object's current state
The second monitor rather "piggybacks"
on the object's own state
Key point (again)
The first monitor observes method calls to
deduce the object's current state
The second monitor rather "piggybacks"
on the object's own state
PIGGYBACK RUNTIMEMONITORING
On an Iterator i, method i.next() should
only be called...
On an Iterator i, method i.next() should
only be called...
if it immediately
follows i.hasNext()
On an Iterator i, method i.next() should
only be called...
if it immediately
follows i.hasNext()
if it has more elements
to enumerate
On an Iterator i, method i.next() should
only be called...
if it immediately
follows i.hasNext()
if it has more elements
to enumerate
history-based
On an Iterator i, method i.next() should
only be called...
if it immediately
follows i.hasNext()
if it has more elements
to enumerate
history-based state-based
Sequences ofmethod calls
Sequences ofmethod calls
Object'sstate
Wait a minute...
Does it work all
the time?
Wait a minute...
Let ~ be an equivalence relation between
two sequences m, m' ∈ A* of method
calls
Let ~ be an equivalence relation between
two sequences m, m' ∈ A* of method
calls
m ~ m' ⇔ for every m'' ∈ A*,
both mm'' and m'm'' are
either valid or invalid
[m] = equivalence class of m
Let S = { [m] : m ∈ A*}.
The relation ~ induces a transition
function ω : S × A → S ; namely
ω([m],m) = [mm]
C behaves like a deterministic
state machine O = 〈S,[ε],ω〉
a
b
b
ac
a
M
[m ]0
[m ]1
[m ]2[m ]4
[m ]3
[m ]5
[m ]6
[m ]7
[m ]8
[m ]9
[m ]10
[m ]11
[m ]12
[m ]12
a b
a
b
b
a
a
bc
b
c
ε
a
a
a
[m
a
a
b
b
ε
εO
a
b
b
ac
a
M
[m ]0
[m ]1
[m ]2[m ]4
[m ]3
[m ]5
[m ]6
[m ]7
[m ]8
[m ]9
[m ]10
[m ]11
[m ]12
[m ]12
a b
a
b
b
a
a
bc
b
c
ε
a
a
a
[m
a
a
b
b
ε
εO
a
b
b
ac
a
M
[m ]0
[m ]1
[m ]2[m ]4
[m ]3
[m ]5
[m ]6
[m ]7
[m ]8
[m ]9
[m ]10
[m ]11
[m ]12
[m ]12
a b
a
b
b
a
a
bc
b
c
ε
a
a
a
[m
a
a
b
b
ε
εO
a
b
b
ac
a
MAll states of Oare accounted for
No state of Ohas two colors
[m ]0
[m ]1
[m ]2[m ]4
[m ]3
[m ]5
[m ]6
[m ]7
[m ]8
[m ]9
[m ]10
[m ]11
[m ]12
[m ]12
a b
a
b
b
a
a
bc
b
c
ε
a
a
a
[m
a
a
b
b
ε
εO
a
b
b
ac
a
MAll states of Oare accounted for
No state of Ohas two colors
(δ is total)
(otherwise O and Mdisagree on some m)
[m ]0
[m ]1
[m ]2[m ]4
[m ]3
[m ]5
[m ]6
[m ]7
[m ]8
[m ]9
[m ]10
[m ]11
[m ]12
[m ]12
a b
a
b
b
a
a
bc
b
c
ε
a
a
a
[m
a
a
b
b
ε
εO
a
b
b
ac
a
MAll states of Oare accounted for
No state of Ohas two colors
(δ is total)
(otherwise O and Mdisagree on some m) (just take the color!)
There exists a mappingH : S→Q such that H is a graph homomorphism 〈S,ω〉 → 〈Q,δ〉
Wait for method call m ∈ AFetch s
Compute q = δ(H(s), m)
If q = then raise error
Else goto 1
1
2
3
4
5
Wait for method call m ∈ AFetch s
Compute q = δ(H(s), m)
If q = then raise error
Else goto 1
1
2
3
4
5
Memory-less (nothing to persist
between calls)
Only monitor calls that may
lead to
Collection of 13 different monitoring
properties in papers from 2001-2011
(java.util package: the “monitoring canon”)
Collection of 13 different monitoring
properties in papers from 2001-2011
(java.util package: the “monitoring canon”)
Given an Iterator i on a Collection c, i.next()
cannot follow any of c.add(), c.delete(), c.set()
Given an Iterator i, i.remove() should not be called
twice without a call to i.next() in between
Don’t use an InputStreamReader after its
InputStream was closed
. . .
Source code analysis of the OpenJDK 6
implementation of Java. Of the 13 properties
in the canon:
Source code analysis of the OpenJDK 6
implementation of Java. Of the 13 properties
in the canon:
10 can be piggyback-monitored
Source code analysis of the OpenJDK 6
implementation of Java. Of the 13 properties
in the canon:
10 can be piggyback-monitored
Remaining 3 either...
Source code analysis of the OpenJDK 6
implementation of Java. Of the 13 properties
in the canon:
10 can be piggyback-monitored
Remaining 3 either...
don't distinguish between
valid/invalid calls
Source code analysis of the OpenJDK 6
implementation of Java. Of the 13 properties
in the canon:
10 can be piggyback-monitored
Remaining 3 either...
don't distinguish between
valid/invalid calls
violate deterministic condition
Wait a minute...
How hard is it to
obtain H and s?
Wait a minute...
The graph homomorphismproblem
Finding an homomorphism H between
two directed graphs G and G' is NP-
complete.
The graph homomorphismproblem
Finding an homomorphism H between
two directed graphs G and G' is NP-
complete.
The graph homomorphismproblem
(Known result)
Finding an homomorphism H between
two directed graphs G and G' is NP-
complete.
The graph homomorphismproblem
(Known result)
Checking if a known H “works” is
polynomial⇒
Wait a minute...
class C int x; B b;...
Wait a minute...
class B float w; A a; int j; D d;
class C int x; B b;...
Wait a minute...
class A float z; int k;
class B float w; A a; int j; D d;
class C int x; B b;...
Wait a minute...
class D ...
class A float z; int k;
class B float w; A a; int j; D d;
class C int x; B b;...
Wait a minute...
class D ...
class A float z; int k;
class B float w; A a; int j; D d;
class C int x; B b;...
Wait a minute...
class D ...
class A float z; int k;
class B float w; A a; int j; D d;
class C int x; B b;...
How many fields are
involved?
Wait a minute...
Propert y REMOVE
Propert y REMOVE
Traditional version
Given an Iterator i, i.remove() should
not be called twice without a call to
i.next() in between
Propert y REMOVE
Traditional version
Given an Iterator i, i.remove() should
not be called twice without a call to
i.next() in between
Piggyback version
Given an Iterator i, in every call to
i.remove(), it must hold that i.lastRet
must not be equal to −1
Propert y REMOVE
Traditional version
Given an Iterator i, i.remove() should
not be called twice without a call to
i.next() in between
Piggyback version
Given an Iterator i, in every call to
i.remove(), it must hold that i.lastRet
must not be equal to −1
member fieldinvolved
1
Propert y SAFE NUME
Propert y SAFE NUME
Traditional version
Given an Iterator i on a Collection c,
i.next() cannot follow any of c.add(),
c.delete(), c.set(), etc.
Propert y SAFE NUME
Traditional version
Given an Iterator i on a Collection c,
i.next() cannot follow any of c.add(),
c.delete(), c.set(), etc.
Piggyback version
Given an Itr i, in every call to i.next(),
it must hold that i.expectedModCount is
equal to i$0.modCount
Propert y SAFE NUME
Traditional version
Given an Iterator i on a Collection c,
i.next() cannot follow any of c.add(),
c.delete(), c.set(), etc.
Piggyback version
Given an Itr i, in every call to i.next(),
it must hold that i.expectedModCount is
equal to i$0.modCount
member fieldsinvolved
2
With the OpenJDK 6, the 10 piggyback
properties involved at most _______
primitive member fields inside a class
With the OpenJDK 6, the 10 piggyback
properties involved at most _______
primitive member fields inside a class
2
With the OpenJDK 6, the 10 piggyback
properties involved at most _______
primitive member fields inside a class
at a level of nesting of at most _______
2
With the OpenJDK 6, the 10 piggyback
properties involved at most _______
primitive member fields inside a class
at a level of nesting of at most _______
2
1
With the OpenJDK 6, the 10 piggyback
properties involved at most _______
primitive member fields inside a class
at a level of nesting of at most _______
2
1
H is simple and shallow
With the OpenJDK 6, the 10 piggyback
properties involved at most _______
primitive member fields inside a class
at a level of nesting of at most _______
2
1
...but many fields are private and have
no accessor methods ☹
H is simple and shallow
Propert y SAFE ILE EADERF
Traditional version
Don’t use an InputStreamReader after its
InputStream was closed
Piggyback version
Given an InputStreamReader r, in every
call to r.read(), it must hold that
r.sd.isOpen is true
R
private
Propert y SAFE ILE EADERF
Traditional version
Don’t use an InputStreamReader after its
InputStream was closed
Piggyback version
Given an InputStreamReader r, in every
call to r.read(), it must hold that
r.sd.isOpen is true
R
class InputStreamReader {
private InputStream sd;
public boolean isStreamOpen() { return sd.isOpen(); }
...
}
class InputStream {
private boolean isOpen;
public boolean isOpen() { return isOpen; }
...
}
MISSING
class InputStreamReader {
private InputStream sd;
public boolean isStreamOpen() { return sd.isOpen(); }
...
}
class InputStream {
private boolean isOpen;
public boolean isOpen() { return isOpen; }
...
}
MISSING
MISSING
class InputStreamReader {
private InputStream sd;
public boolean isStreamOpen() { return sd.isOpen(); }
...
}
class InputStream {
private boolean isOpen;
public boolean isOpen() { return isOpen; }
...
}
class D ...
class A float z; int k;
class B float w; A a; int j; D d;
class C int x; B b;...
Wait a minute...
class D ...
class A float z; int k;
class B float w; A a; int j; D d;
class C int x; B b;...
private int x;
private int j;
private D d;
private
What about information
hiding?
Wait a minute...
“...one begins [to decompose a system] with a
list of difficult design decisions or design
decisions that are likely to change. Each
module is then designed to hide such a
decision from the others.”
― David Lodge Parnas, 1972
class InputStreamReader {
private InputStream sd;
public boolean isStreamOpen() { return sd.isOpen(); }
...
}
class InputStream {
private boolean isOpen;
public boolean isOpen() { return isOpen; }
...
}
Difficult design decision?
Likely to change?
class InputStreamReader {
private InputStream sd;
public boolean isStreamOpen() { return sd.isOpen(); }
...
}
class InputStream {
private boolean isOpen;
public boolean isOpen() { return isOpen; }
...
}
The member fields of C involved in
the homomorphism H should have
the same read access visibilit y as C
The member fields of C involved in
the homomorphism H should have
the same read access visibilit y as C
design for monitoring
The member fields of C involved in
the homomorphism H should have
the same read access visibilit y as C
design for monitoring (or just for correct use)
Wait a minute...
Is overhead actually
reduced ?
OVERHEAD
Wait a minute...
Runtime monitoring of the 10 piggyback
properties on the DaCapo benchmark
Set of open source, real world applications
with non-trivial memory load
Same benchmark used by past monitoring
papers
Benchmark error margin ≈ 3%
Benchmark error margin ≈ 3%
Piggyback memory overhead ≈ 0%
Benchmark error margin ≈ 3%
Piggyback memory overhead ≈ 0%
Piggyback runtime overhead ≈ 3%
Benchmark error margin ≈ 3%
Piggyback memory overhead ≈ 0%
Piggyback runtime overhead ≈ 3%
Benchmark
pmd
avrora
Runtime overhead (%)
Piggyback
≈ 0%
≈ 1%
Highlights:
Benchmark error margin ≈ 3%
Piggyback memory overhead ≈ 0%
Piggyback runtime overhead ≈ 3%
Benchmark
pmd
avrora
Runtime overhead (%)
Piggyback
≈ 0%
≈ 1%
Highlights:
Classical
≈ 123%
≈ 118%
EpilogueConclusion and future work
Piggyback runtime monitoring uses an
object's own, existing member fields as an
encoding of the monitor's state
Under some conditions, this encoding is
guaranteed to exist
Stateful properties become stateless
properties ; no data to persist between calls
Emprical evidence on real-world
benchmarks on the OpenJDK 6 reveal
that:
Emprical evidence on real-world
benchmarks on the OpenJDK 6 reveal
that:
Most properties are piggyback-
compatible
Emprical evidence on real-world
benchmarks on the OpenJDK 6 reveal
that:
Most properties are piggyback-
compatible
H involves few variables
Emprical evidence on real-world
benchmarks on the OpenJDK 6 reveal
that:
Most properties are piggyback-
compatible
H involves few variables
Runtime overhead is reduced over
classical monitors
Future work and open questions:
Use annotations instead of aspects
Find H automatically
Verify a given H statically
Extend to methods with arguments
Relax hypotheses
Piggyback and classical monitors are
complementary
Piggyback and classical monitors are
complementary
The choice of a classical monitor should be
justified, not taken for granted
Piggyback and classical monitors are
complementary
The choice of a classical monitor should be
justified, not taken for granted
Is overhead really what matters?
The EndThank you !
