Michael Bond Kathryn McKinley

The University of Texas at Austin

http://www.codeproject.com/KB/showcase/IfOnlyWedUsedANTSProfiler.aspx

Driverless truck 10,000 lines of C#

Leak: past obstacles remained reachableNo immediate symptoms

“This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles.”

Quick “fix”: restart after 40 minutesEnvironment sensitive

More obstacles in deployed setting Unresponsive after 28 minutes

Driverless truck 10,000 lines of C#

Leak: past obstacles remained reachableNo immediate symptoms

“This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles.”

Quick “fix”: restart after 40 minutesEnvironment sensitive

More obstacles in deployed setting Unresponsive after 28 minutes

Different environments & inputs different behavior

Uncertainty in deployed systems

Hard to fix before

deployment

Deployed systems have leaks Critical systems need immediate help

Leak pruning tolerates bad effects Reclaims memory automatically High precision & low overhead Bounds resources Preserves semantics

Why tolerate leaks

Why leaks are so bad

How leak pruning works

How leak pruning predicts leaked objects

Evaluation

Memory leaks are a real problem Managed languages do not eliminate

them

Memory leaks are a real problem Managed languages do not eliminate

them

Reachable

Unrea

chab

le

Memory leaks are a real problem Managed languages do not eliminate

them

LiveLive

ReachableDead

Memory leaks are a real problem Managed languages do not eliminate

them

LiveLive

Reachable

Dead

Memory leaks are a real problem Managed languages do not eliminate

them

LiveLive

Reach

able

Dead

Memory leaks are a real problem Managed languages do not eliminate

them Slow & crash real programs

LiveLiveDead

Memory leaks are a real problem Managed languages do not eliminate

them Slow & crash real programs Unacceptable for some applications

Fixing leaks is hard Leaks take time to materialize Failure far from cause

Is it a memory leak or a case of stale memory ?

Is it correct to always consider stale memory as equivalent to unreachable memory ?

Why tolerate leaks

Why leaks are so bad

How leak pruning works

How leak pruning predicts leaked objects

Evaluation

Garbage collection based on liveness

LiveLive

Reachable

Dead

Garbage collection based on liveness Take action when memory exhausted

LiveLive

Reachable

Dead

Garbage collection based on liveness Take action when memory exhausted

LiveLive

ReachableDead

Garbage collection based on liveness Take action when memory exhausted

LiveLive

DeadThrow OOM

errorThrow OOM

error

Out of memor

y!

Out of memor

y!

Garbage collection based on liveness Take action when memory exhausted

LiveLive

Dead

Out of memor

y!

Out of memor

y!

Throw OOM error

Throw OOM error

Reclaim some

objects

Reclaim some

objects

Garbage collection based on liveness Reclaim predicted dead objects

Reclaim

ed

LiveLive

Garbage collection based on liveness Reclaim predicted dead objects

Reclaim

ed

LiveLive

AB

Garbage collection based on liveness Reclaim predicted dead objects

LiveLive

A?

Garbage collection based on liveness Reclaim predicted dead objects Poison references to reclaimed objects

LiveLive

AX

Garbage collection based on liveness Reclaim predicted dead objects Poison references to reclaimed objects

LiveLive

AXThrow InternalError with OOMError attached

Garbage collection based on liveness Reclaim predicted dead objects Poison references to reclaimed objects

LiveLive

AXThrow InternalError with OOMError attached

Worst case: defers fatal

errors

Worst case: defers fatal

errors

Best case: keeps leaky programs running much

longer or indefinitely

Best case: keeps leaky programs running much

longer or indefinitely

Instead of returning the error can we return some other object, to keep program going ?

Eg DieHardWhat about semantics in that case ?

INACTIVE

OBSERVE

SELECT

PRUNE

Heap not full

<50% Heap filled

>50%

Heap not full

Heap full

Why tolerate leaks

Why leaks are so bad

How leak pruning works

How leak pruning predicts leaked objects

Evaluation

Predicting the future Leaked objects not used again Highly stale objects likely leaked

Alternative: offload to disk [Melt, Bond & McKinley ’08] [LeakSurvivor, Tang et al. ’08] [Panacea, Goldstein et al. ’07, Breitgand et al. ’07] Tolerates mispredictions Eventually exhausts disk (if disk at all)

Predicting the future Leaked objects not used again Highly stale objects likely leaked

Alternative: offload to disk [Melt, Bond & McKinley ’08] [LeakSurvivor, Tang et al. ’08] [Panacea, Goldstein et al. ’07, Breitgand et al. ’07] Tolerates mispredictions Eventually exhausts disk (if disk at all)

Need high precisionOne misprediction: program terminates

Identify references to prune Roots of leaked data structures Categorize by reference type

ParserInfoPreparedStatement

Identify references to prune Roots of leaked data structures Categorize by reference type

Criteria Highly stale references

▪ More stale than most stale instance accessed previously

ParserInfoPreparedStatement

MaxS&U

2-4 GCs

Identify references to prune Roots of leaked data structures Categorize by reference type

Criteria Highly stale references

▪ More stale than most stale instance accessed previously

Reference type keeping most bytes reachable

ParserInfoPreparedStatement

MaxS&U

2-4 GCs

Size

132MB

Identify references to prune Roots of leaked data structures Categorize by reference type

Criteria Highly stale references

▪ More stale than most stale instance accessed previously

Reference type keeping most bytes reachablePiggyback on GC: two-phase transitive closure

ParserInfoPreparedStatement

MaxS&U

2-4 GCs

Size

132MB

Based on colleague’s JDBC application

Leak: SQL statements remain in set

HashSetParser

Info

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

enqueued

enqueued

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

enqueued

enqueued

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32

enqueued

enqueued

enqueued

enqueued

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

enqueued

enqueued

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

0

0

0

enqueued

enqueued

enqueued

enqueued

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

enqueued

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

enqueued

enqueued

enqueued

enqueued

20

0

0

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

enqueued

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

60

enqueued

enqueued

enqueued

enqueued

0

0

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

enqueued

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

572

enqueued

enqueued

enqueued

enqueued

0

0

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

572

enqueued

enqueued

enqueued

enqueued

0

0

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

572

enqueued

enqueued

enqueued

enqueued

244

0

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

420M

180M

0

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

420M

180M

0

HashSet

…

……

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

ParserInfo

ResultSet

HashEntry

PreparedStatement

Field[]

byte[][] byte[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

420M

180M

0

HashSet

…

……

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

420M

180M

0

HashSet

…

……

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

0

0

0

HashSet

…

……

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

0

0

0

enqueued

enqueued

enqueued

HashSet

…

……

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

50M

220M

0

HashSet

…

……

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

50M

220M

0

HashSet

…

……

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

ResultSet

HashEntry

PreparedStatement

Field[]

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

50M

220M

0

HashSet

…

…

HashEntry

PreparedStatement

HashEntry

PreparedStatement

HashEntry

PreparedStatement

HashEntry PreparedStatement

PreparedStatement ParserInfo

PreparedStatement ResultSet

MaxS&U

0-1

0-1

16-32Bytes

50M

220M

0

HashSet

…

…

HashEntry

PreparedStatement

HashEntry

PreparedStatement

HashEntry

PreparedStatement

Algorithm efficiently selects

references to consistently unused data structures based on type

Why tolerate leaks

Why leaks are so bad

How leak pruning works

How leak pruning predicts leaked objects

Evaluation

Leak pruning added to Jikes RVM 2.9.2

http://www.jikesrvm.org/Research+Archive

Performance stress test Non-leaking programs: DaCapo & SPEC

benchmarks Low overhead

▪ 3% (Core 2) or 5% (Pentium 4)▪ Primarily read barriers

Low footprint &often helps a

lot all-the-time

deployed use

INACTIVE

OBSERVE

SELECT

PRUNE

Heap not full

<50% Heap filled

>50%

Heap not full

Heap full

What about bursty leaks ?Can we have dynamically adjustable

limit for switching on Leak Pruning ?Can we dynamically switch off Leak

Pruning ?

What if stale objects interleaved with active objects ?

Is it useful to find source of these leaks ? If yes then how ?