Dennis Byrnedennisbyrne@apache.org

Memory Barriers

Defining the Problem

Object connection = null; boolean initialized = false;

// thread 1 writes twice // thread 2 reads twice

connection = new Connection(); if(initialized)initialized = true; connection.use();

NullPointerException ?

Root Cause: Memory Latency

• Processors work hard to avoid memory latency– memory operations (reads & writes) are re-ordered

• This is not a problem when …– data is local and/or immutable– there is only single processor

• People do this also …

Introduction

Memory barriers, or fences, are a set of processor instructions

used to apply ordering limitations on read and write operations.

Visibility is “kind of” important

– The Java Memory Model– Erlang send operators– Retlang and Jetlang Channels– C++ atomics– Scala Actors– Every semaphore, mutex, or atomic operation

Consecutive Volatile Writes on Itanium 2

1 adds r37=592,r36;; ;...0b284149 04212 st4.rel [r37]=r39 ;...00389560 23803 adds r36=596,r36;; ;...841125444 st1.rel [r36]=r0 ;...09000048 a0115 mf ;...00000044 00006 nop.i 0x0;; ;...000400007 mov r12=r33 ;...00600042 00218 mov.ret b0=r35,0x2000000001de81e0

The other side of the protocol ….

Consecutive Volatile Reads on Itanium 2

1 adds r37=597,r36;; ;...841125542 ld1.acq r38=[r37];; ;...0b30014a a0103 nop.m 0x0 ;...00000002 00c04 sxt1 r38=r38;; ;...005130045 cmp4.eq p0,p6=0,r38 ;...1100004c 86396 nop.i 0x0 ;...00000002 00037 br.cond.dpnt.many 0x2000000001de8220;;

One side of the protocol ….

Implicit Memory Barriers

mov 0x160(%edi),%edi ;...8bbf6001 0000mov %ecx,%edi ;...8bf9add $0x8,%edi ;...83c708lock cmpxchg %esi,(%edi) ;...f00fb137mov $0x1,%eax ;...b8010000 00

Atomic CAS operation on x86

lock cmpxchg serializes pending memory operations

Avoiding Memory Barriers

Atomic CAS on a VMWare image with one processor:

add $0x8,%edi ;...83c708cmpxchg %esi,(%edi) ;...0fb137mov $0x1,%eax ;...b8010000 00

Consecutive volatile reads in Java on SPARC:

ld [ %l1 + 0x150 ], %i0 ;...f0046150sethi %hi(0xff3fc000), %l0 ;...213fcff0ld [ %l0 ], %g0 ;...c0042000ret ;...81c7e008

Memory Barriers

ThanksDennis Byrne – DRW Trading

dennisbyrne@apache.org