© Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20041 A Rapid...

© Richard Jones, Eric Jul, 1999-2004 mmnet GC & MM Summer School, 20-21 July 2004 1

A Rapid Introduction to Garbage CollectionA Rapid Introduction to Garbage Collection

Richard JonesComputing Laboratory

University of Kent at Canterbury

mm-net Garbage Collection & Memory Management Summer School

Tuesday 20 July 2004

© Richard Jones, 2004.All rights reserved.


PART 1: IntroductionPART 1: Introduction

Motivation for garbage collection

What to look for

Motivation for garbage collection

What to look for


Why garbage collect? Why garbage collect?

Finite storage requirement• computer have finite, limited storage

Language requirement• many OO languages assume GC, e.g. allocated objects

may survive much longer than the method that created them

Problem requirement• the nature of the problem may make it very hard/impossible

to determine when something is garbage


Why automatic garbage collection?

Why automatic garbage collection?

Because human programmers just can’t get it right.Either

too little is collected leading to memory leaks, or too much is collected leading to broken programs.

Explicit memory management conflicts with the software engineering principles of abstraction and modularity.

It’s not a silver bullet

• Some memory management problems cannot be solved using automatic

GC, e.g. if you forget to drop references to objects that you no longer need.

• Some environments are inimical to garbage collection– embedded systems with limited memory– hard real-time systems


PART 2: The BasicsPART 2: The Basics

• What is garbage?

• The concept of liveness by reachability

• The basic algorithms

• The cost of garbage collection

• What is garbage?

• The concept of liveness by reachability

• The basic algorithms

• The cost of garbage collection


What is garbage?What is garbage?

Almost all garbage collectors assume the following definition of live objects called liveness by reachability: if you can get to an object, then it is live.

More formally: An object is live if and only if:

it is referenced in a predefined variable called a root,

or

it is referenced in a variable contained in a live object

(i.e. it is transitively referenced from a root).

Non-live objects are called dead objects, i.e. garbage.


RootsRoots

Objects and references can be considered a directed graph.Live objects are those reachable from a root. A process executing a computation is called a mutator — it simply modifies the object graph dynamically.

Determining roots of a computation is, in general, language-dependent.

In common language implementations roots include• words in the static area

• registers

• words on the execution stack that point into the heap.


The basic algorithmsThe basic algorithms

• Reference counting: Keep a note on each object in your garage, indicating the number of live references to the object. If an object’s reference count goes to zero, throw the object out (it’s dead).

• Mark-Sweep: Put a note on objects you need (roots). Then recursively put a note on anything needed by a live object. Afterwards, check all objects and throw out objects without notes.

• Mark-Compact: Put notes on objects you need (as above). Move anything with a note on it to the back of the garage. Burn everything at the front of the garage (it’s all dead).

• Copying: Move objects you need to a new garage. Then recursively move anything needed by an object in the new garage. Afterwards, burn down the old garage (any objects in it are dead)!


ST

R root

ST

R root

Update(left(R), S)

Reference countingReference countingThe simplest form of garbage collection is reference counting.

Basic idea: count the number of references from live objects.

Each object has a reference count (RC) • when a reference is copied, the referent’s RC is incremented

• when a reference is deleted, the referent’s RC is decremented

• an object can be reclaimed when its RC = 0


Advantages of reference counting

Advantages of reference counting

Simple to implement

Costs distributed throughout program

Good locality of reference: only touch old and new targets' RCs

Works well because few objects are shared and many are short-lived

Zombie time minimized: the zombie time is the time from when an object becomes garbage until it is collected

Immediate finalisation is possible (due to near zero zombie time)

OHP


Disadvantages of reference countingDisadvantages of reference counting

Not comprehensive (does not collect all garbage):cannot reclaim cyclic data structures

High cost of manipulating RCs: cost is ever-present even if no garbage is collected

Bad for concurrency — need Compare&Swap

Tightly coupled interface to mutator

High space overheads

Recursive freeing cascade

OHP


Mark-SweepMark-Sweep

Mark-sweep is a tracing algorithm — it works by following (tracing) references from live objects to find other live objects.

Implementation:Each object has a mark-bit associated with it.

There are two phases:• Mark phase: starting from the roots, the graph is traced and

the mark-bit is set in each unmarked object encountered. At the end of the mark phase, unmarked objects are

garbage.

• Sweep phase: starting from the bottom, the heap is swept – mark-bit not set: the object is reclaimed– mark-bit set: the mark-bit is cleared


A simple mark-sweep exampleA simple mark-sweep example

root

mark-bit

0

1

2


Comprehensive: cyclic garbage collected naturally

No run-time overhead on pointer manipulations

Loosely coupled to mutator

Does not move objects• does not break any mutator invariants

• optimiser-friendly

• requires only one reference to each live object to be discovered (rather than having to find every reference)

Advantages of mark-sweepAdvantages of mark-sweepOHP


Disadvantages of mark-sweepDisadvantages of mark-sweep

Stop/start nature leads to disruptive pauses and long zombie times.

Complexity is O(heap) rather than O(live) • every live object is visited in mark phase• every object, alive or dead, is visited in sweep phase

Degrades with residency (heap occupancy)• the collector needs headroom in the heap to avoid thrashing

Fragmentation and mark-stack overflow are issues

Tracing collectors must be able to find roots (unlike reference counting)

OHP


Fast allocation?Fast allocation?

Problem: Non-moving memory managers fragment the heap

• mark-sweep

• reference counting

A compacted heap

• offers better spatial locality, e.g. better virtual memory and cache performance

• allows fast allocation

– merely bump a pointer


Registers

freescan

Tospace

From space

C

F GD E

B

A

Registers

freescan

Tospace

From space

C

F GD E

B

A

A'

A'

copy root and update pointer,

leaving forwarding address

Registers

freescan

Tospace

From space

C

F GD E

B

A

B'

A'

A'

B' C'

C'

scan A'copy B and C,

leaving forwarding addresses

Registers

freescan

Tospace

From space

C

F GD E

B

A

D' E'B' C'

B' C'

A'

A'

D' E'

scan B'copy D and E,


Registers

freescan

Tospace

From space

C

F GD E

B

A

D' E'B' C'

B' C'

A'

A' F'

G'

D' E' F' G'

scan C'copy F and G,


Registers

freescan

Tospace

From space

C

F GD E

B

A

F' G'

D' E'

D' E'

B' C'

B' C'

A'

A' F'

G'

scan D' and E'nothing to do

Registers

freescan

Tospace

From space

C

F GD E

B

A

F' G'

D' E'

D' E'

B' C'

B' C'

A'

A' F'

G'

scan F'use A's forwarding address

Registers

freescan

Tospace

From space

C

F GD E

B

A

F' G'

D' E'

D' E'

B' C'

B' C'

A'

A' F'

G'

scan G'nothing to do

Registers

freescan

Tospace

From space

C

F GD E

B

A

F' G'

D' E'

D' E'

B' C'

B' C'

A'

A' F'

G'

scan=freeso collection is complete

Co

pyi

ng

GC

Exa

mp

leC

op

yin

g G

C E

xam

ple


Disadvantages of copying GCDisadvantages of copying GC

Stop-and-copy may be disruptiveDegrades with residency

Requires twice the address space of other simple collectors• touch twice as many pages• trade-off against fragmentation

Cost of copying large objectsLong-lived data may be repeatedly copied

All references must be updatedMoving objects may break mutator invariants

Breadth-first copying may disturb locality patterns


Mark-compact collectionMark-compact collection

Mark-compact collectors make at least two passes over the heap after marking

• to relocate objects

• to update references (not necessarily in this order)

Issues• how many passes?

• compaction style

– sliding: preserve the original order of objects

– linearising: objects that reference each other are placed adjacently (as far as possible)

– arbitrary: objects moved without regard for original order or referential locality


Cost metricsCost metrics

Many cost metrics can be interesting (albeit not necessarily at the same time). These cost metrics cover different types of concerns that may apply. The metrics are partially orthogonal, partially overlapping, and certainly also partially contradictory.

In general it is not possible to identify one particular metric as the most important in all cases — it is application dependent.

Because different GC algorithms emphasise different metrics, it is also, in general, not possible to point out one particular GC algorithm as “the best”.

In the following, we present the most important metrics to consider when choosing a collector algorithm.


GC MetricsGC Metrics

Execution time• total execution time• distribution of GC execution

time• time to allocate a new

object

Memory usage• additional memory

overhead• fragmentation• virtual memory and cache

performance

Delay time• length of disruptive

pauses• zombie times

Other important metrics• comprehensiveness• implementation simplicity

and robustness


PART 3: Generational GCPART 3: Generational GC


The generational hypothesisThe generational hypothesis

Weak generational hypothesis “Most objects die young” [Ungar, 1984]

It is common for 80-95% objects to die before a further megabyte has been allocated

• 95% of objects are ‘short-lived’ in many Java programs

• 50-90% of CL and 75-95% of Haskell objects die before they are 10kb old

• SML/NJ reclaims 98% of any generation at each collection

• Only 1% Cedar objects survived beyond 721kb of allocation


Semi-space copying vs. generational GC

Semi-space copying vs. generational GC

newgeneration

oldgeneration

Heapoccupancy

Time

garbagecollection

pause

semi-spaces

Time

New data

Copied data


ExampleExample

c

b

a

root set

Old generation New generation

S

c

b

a

root set


S

Update with pointer to a; request new object; allocation fails

c

b

a

R

root set


S

perform minor collection

c

b

a

R

root set


S

further updates, allocation, etc...where are the roots for the new generation?


Not a universal panaceaNot a universal panacea

Generational GC is a successful strategy for many but not all programs. There are common examples of programs that do not obey the weak generational hypothesis.

It is common for programs to retain most objects for a long time and then to release them all at the same time.

Generational GC imposes a cost on the mutator:

pointer writes become more expensive

certain classes of program may thrash this overhead. – e.g. a program may repeatedly process a large long-lived array

of heap-allocated data, updating many, if not all, of the array's slots at each iteration.

need to copy objects.


Roots of the new generationRoots of the new generation

c

b

a

R

root set


S


Issues raised by generational garbage collection

Issues raised by generational garbage collection

Old-young pointers give rise to roots for the young generation: how can these roots be discovered?

Garbage in older generations cannot be reclaimed by minor collections: how can this be minimised?

When should surviving objects be promoted to the next generation?

How do we record object ages?

How should generations be organised?

OHP


Problem:Intergenerational pointers

Problem:Intergenerational pointers

We can collect the young generation on its own (minor collection)

Old-young pointers give rise to roots for the young generation• such pointers are comparatively rare• they arise from destructive pointer writes• these assignments can be trapped with a write barrier

Young-old pointers are common• but not a problem if we collect younger generation

whenever we collect an older one (major collection)

OHP


Problem:Tenured garbage

Problem:Tenured garbage

Garbage in older generations cannot be reclaimed by minor collections

Root set


Tenured garbage also causes the retention of young objects it references (nepotism)


Multiple generationsMultiple generations

Allow youngest generation to be kept small hence, short pauses for minor collections

and good locality

– try to tune to size of cache?

Filter objects prematurely promoted allows more time to die

prevents garbage reaching oldest generation

But, more complex

intermediate collections may also be disruptive


Promotion ratePromotion rate

Promotion rate depends on the number of minor collections that an object must survive.

Copy count = 1? en masse promotion: all young objects promoted

allows simple heap organisation, but

young objects have too little opportunity to die

promotion rate 50-100% higher than necessary

Copy count = 2? denies promotion to recently created objects

may reduce survivors by a factor of 2, but

may increase copying costs by less than half

Copy count > 2?

gives diminishing returns


Generation organisationsGeneration organisations

1) One semi-space per generation, en masse promotion no need to record ages youngest region recycled at each GC — good locality

requires multiple generations

causes more write-barrier traps

2) Divide generation into a creation space and an aging space• aging space holds survivors from creation space

– must be organised into semi-spaces since objects may be held for more than one scavenge

– but semi-spaces can be kept small• creation space can be reused at each collection

– locality benefit– keep in physical memory (or cache)


Other organisationsOther organisations

Bucket-brigades• divide each generation into sub-regions: determine age by a

pointer comparison rather than recording per-object ages.

Large Object Areas• managed by non-copying GC

Older-First Collection• assumes that recently allocated objects need time to die.

• objects laid out in allocation order

• collector processes objects in older-first order.

Beltway• Beltway is a flexible framework that exploits and separates object

age and incrementality. It encompasses all other copying collectors.


Inter-generational pointersInter-generational pointers

Need to discover these as they are roots for younger generations

Scan older generations?no cost to mutator

requires more scanning at GC time (but scanning is faster than tracing and has better locality)

Trap writes?• IGPs caused through pointer stores and promotion• Software or hardware techniques• Issues

– cost to the mutator (time, code size)– space overhead– discovery at scavenge-time


Roots of the new generationRoots of the new generation

c

b

a

R

root set


S

OHP


Write barriersWrite barriers

Write barriers can be implemented in software or with operating system support.

Software• have the compiler emit extra instructions

Operating system• use memory protection mechanism• use paging mechanism dirty bits


Trapping pointer writesTrapping pointer writes

Problem: need to trap pointer writes that may create old-young pointers

• don’t need to trap stores to stack or initialising stores– can initialising stores be distinguished from other pointer

writes?

• writes relatively uncommon (but language dependent)– Lisp: 5-10% of memory references are non-initialising pointer

stores– SML/NJ: frequency of pointer stores is less than 1%

• cost also depends on whether language is interpreted or not

There are 2 ways to do this.

OHP


Remembered setsRemembered sets

Keep address of old object containing pointer to young one• question: address of object or address of slot in object?

Scanning cost dependent on size of set (not number of pointer stores) and size of objects (if slot addresses not remembered)

Old generation Young generation

Remembered set


Card tablesCard tables

Aims: a faster write barriersmall memory costportability

• Divide heap into small cards (e.g. 128 bytes table < 1% heap)• Set a bit in the card table unconditionally whenever a word in

heap corresponding to that card is modified• Scan modified cards at scavenge-time

Result• Fast trap — 3 instructions• Cost of scanning proportional to number and size of cards

marked• Difficulty: how to handle objects that span cards?


Generational GC:a summary

Generational GC:a summary

Highly successful for a range of applications reduces pause time to a level acceptable for interactive

applications

improves paging and cache behaviour

reduces the overall cost of garbage collection

Requires a low survival rate, infrequent major collections, low overall cost of write barrier

But generational GC is not a universal panacea. It attempts to improve expected pause time at expense of the worst case

objects may not die sufficiently fast

applications may thrash the write barrier

too many old-young pointers, or very deep execution stacks, may increase pause times


PART 4: The GC InterfacePART 4: The GC Interface


GC InterfaceGC InterfaceProgrammer’s viewpoint:

• Memory management policies and configurations

• When to collect?

• Region in which to allocate

– tracing or reference counting?

– moving or non-moving?

– immortal?

– region sizes

• Programming restrictions

Implementer's viewpoint:• When to collect?

– System.gc()?

– region full?

– other data structures?

• Safe points

• Threads

– synchronisation

– avoiding locks

• Root finding


Root findingRoot finding

Global roots, roots in other regions• static area, class data structures

• card tables, rem-sets; maintained by write-barriers

Stacks• which slots contain references?

• scan conservatively, or

• type accurately

– compiler generates a stack map

– map PC to stack info

– significant fraction of code size but compresses well

– only collect at GC-safe points

– ambiguities (Java jsr problem)


Safe-pointsSafe-points

Before a ‘stop the world’ GC, need to halt all mutator threads at safe-points

• make every instruction a safe point?

• polling (e.g. before method calls, back-branches, etc)

• patching (GC patches in suspensions)

Suspension can account for significant proportion of GC-time.


PART 5: Incremental andconcurrent garbage collection

PART 5: Incremental andconcurrent garbage collection

Incremental/concurrent garbage collection

• runs collector interleaved/in parallel with mutator

• attempts to bound pause time

• many soft real-time solutions

• but no general hard real-time solutions yet

Incremental/concurrent garbage collection

• runs collector interleaved/in parallel with mutator

• attempts to bound pause time

• many soft real-time solutions

• but no general hard real-time solutions yet


TerminologyTerminology

mutator

collector

Incremental collection

idle idle

Concurrent collection

Parallel collection

Suppose GC ‘should’ account for 20% of execution time

Multiple GC threads: GC = 20% time

Single GC thread: 33% time


A B

C

root

A B

C

root

A B

C

root

Asynchronous execution of mutator and collector introduces a coherency problem. For example, in the marking phase

SynchronisationSynchronisation

Update(right(B), right(A))right(A) = nil Update(right(A), right(B))right(B) = nil

Collectormarks A

Collectorscans A

Collectormarks B

Collectorscans B


Tricolour abstractionTricolour abstractionBlackBlack

• object and its immediate descendants have been visited

• GC has finished with black objects and need not visit again.

GreyGrey

• object has been visited but its components may not have been scanned.

• or, for an incremental/concurrent GC, the mutator has rearranged connectivity of the graph.

• in either case, the collector must visit them again.

WhiteWhite

• object is unvisited and, at the end of the phase, garbage.

A collection terminates when no grey objects remain, i.e. all live objects have been blackened.

OHP


ExampleExample

root

mark-bit

grey set


Two ways to prevent disruption

Two ways to prevent disruption

There are two ways to prevent the mutator from interfering with a collection by writing white pointers into black objects.

1) Ensure the mutator never sees a white object• when mutator attempts to access a white object,

the object is visited by the collector

• protect white objects with a read-barrier

2) Record where mutator writes black-white pointers, so that the GC can (re)visit objects

• protect objects with a write-barrier


Write-barrier methodsWrite-barrier methods

To falsely reclaim an object, two conditions must hold: a pointer to the white object is written into a black object

and furthermore, this must be the only reference to the white object the original reference to the white object is destroyed

If does not hold, there will be at least one path to each reachable white object that passes through a grey object.If does not hold, the white object will still be reachable through the original reference.

Write barrier methods• incremental update methods catch changes to connectivity• snapshot-at-the-beginning methods prevent the loss of the

original reference


IssuesIssues

Barriers lead to floating garbage.

How conservative is an algorithm?• how much garbage is left floating in the heap until the next

collection cycle?

• what is the policy towards new (and often short-lived) objects? what colour are they allocated?

How expensive is the barrier?

How is initialisation achieved? How is termination achieved? In particular

• how are large root sets handled?

• how are threads synchronised?


Best known method was introduced by Dijkstra et al Update (A,C) { *A = C shade(C)}shade(P) { if white(P) colour(P) = grey}

The barrier • traps attempts to install a pointer to a white object into a black

object• incrementally records changes to the shape of the graph• prevents condition arising• but no special action is required when a pointer is deleted

Incremental update methodsIncremental update methods

B C

A

B C

A


Snapshot at the beginningSnapshot at the beginning

Update(A, C) { shade(*A) *A = C}

The barrier • remembers old references

• prevents condition arising

More conservative than IU

Simpler termination: only have to scan roots once, even if no barrier on local variables and temporaries

B C

A

B C

A


Read barrier methodsRead barrier methods

Idea: don't let the mutator see white objects• so it cannot disrupt the collector

• but mutator can access black objects

• Question: should mutator be allowed to see grey objects as well?

Best known is Baker's copying collector. During collection• each mutator read from Fromspace is trapped by read barrier

• writes are OK — they are not trapped

• objects are copied to Tospace, at B

• allocation is made at top of Tospace, at T

Tospace

TBscan

newallocation

copiedobjects

© Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20041 A Rapid...

Documents

Transcript of © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20041 A Rapid...