Testing Concurrent Programs Program Testing and Analysis€¦ · int tmp1 = balance; balance = tmp1...

Program Testing and Analysis:

Testing Concurrent Programs

Prof. Dr. Michael Pradel

Software Lab, TU Darmstadt

Warm-up Quiz

var a = (0.1 + 0.2) + 0.3;var b = 0.1 + (0.2 + 0.3);console.log(a === b);

What does the following code print?

Something elsefalsetrue

Warm-up Quiz

Floating point numbers are representedwith finite precision(not only in JavaScript)

Warm-up Quiz

0.30000000000000004(due to rounding)

Outline

1. Introduction

2. Dynamic Data Race Detection

3. Testing Thread-Safe Classes

4. Exploring Interleavings

Mostly based on these papers:

� Eraser: A Dynamic Data Race Detector for MultithreadedPrograms, Savage et al., ACM TOCS, 1997

� Fully Automatic and Precise Detection of Thread SafetyViolations, Pradel and Gross, PLDI 2012

� Finding and Reproducing Heisenbugs in ConcurrentPrograms, Musuvathi et al., USENIX 2008

Why Bother with Concurrency?

� The free lunch provided by Moore’s law is over� CPU clock speeds stopped to increase around 2005� Instead, multi-core processors became mainstream� Need concurrent programs to make full use of the

hardware

� Many real-world problems are inherentlyconcurrent, e.g.,� Servers must handle multiple concurrent requests� Computations done on huge data often are

”embarrasingly parallel”

hardware

Concurrency Styles

� Message-passing� Popular for large-scale scientific computing, e.g.,

MPI (message-passing interface)� Used in actor concurrency model, e.g., popular in

Erlang and Scala� No shared memory (ideally), all communication via

messages

� Thread-based, shared memory� Multiple concurrently executing threads� All threads access the same shared memory� Synchronize via locks and barriers

Concurrency Styles

� Message-passing� Popular for large-scale scientific computing, e.g.,

MPI (message-passing interface)� Used in actor concurrency model, e.g., popular in

Erlang and Scala� No shared memory (ideally), all communication via

messages

� Thread-based, shared memory� Multiple concurrently executing threads� All threads access the same shared memory� Synchronize via locks and barriers

Focus of this lecture

Example

int a = 0, b = 0;

boolean r = false, t = false;

a = 1;

r = true;

t = r;

b = a;

Thread 1 Thread 2

What does this program mean?

→ Behavior depends on threadinterleaving

Sequential Consistency

Assumption made here:Programs execute under sequential consistency

� Program order is preserved: Each thread’sinstructions execute in the specified order

� Shared memory behaves like a global array:Reads and writes are done immediately

� We assume sequential consistency for the rest ofthe lecture

� Many real-world platforms have more complexsemantics (”memory models”)

What Can Go Wrong?

Common source of errors: Data races

� Two accesses to the same shared memorylocation

� At least one access is a write

� Ordering of accesses is non-deterministic

Example

// bank account

int balance = 10;

// deposit money

int tmp1 = balance;

balance = tmp1 + 5;

// withdraw money

int tmp2 = balance;

balance = tmp2 - 7;

Thread 1 Thread 2

Example

// bank account

int balance = 10;

// deposit money

int tmp1 = balance;

balance = tmp1 + 5;

// withdraw money

int tmp2 = balance;

balance = tmp2 - 7;

Thread 1 Thread 2

Sharedmemorylocation

Example

// bank account

int balance = 10;

// deposit money

int tmp1 = balance;

balance = tmp1 + 5;

// withdraw money

int tmp2 = balance;

balance = tmp2 - 7;

Thread 1 Thread 2

3 races

Example

// bank account

int balance = 10;

// deposit money

int tmp1 = balance;

balance = tmp1 + 5;

// withdraw money

int tmp2 = balance;

balance = tmp2 - 7;

Thread 1 Thread 2

Quiz: What values can balance

have after executing this code?

Example

// bank account

int balance = 10;

// deposit money

int tmp1 = balance;

balance = tmp1 + 5;

// withdraw money

int tmp2 = balance;

balance = tmp2 - 7;

Thread 1 Thread 2

Possible outcomes:balance may be 3, 8, and 15

But: Only 8 is correct

Avoiding Data Races

Use locks to ensure that accesses toshared memory do not interfere

int balance = 10;

acquire(L);

int tmp1 = balance;

balance = tmp1 + 5;

release(L);

acquire(L);

int tmp2 = balance;

balance = tmp2 - 7;

release(L);

Thread 1 Thread 2

Avoiding Data Races

int balance = 10;

acquire(L);

int tmp1 = balance;

balance = tmp1 + 5;

release(L);

acquire(L);

int tmp2 = balance;

balance = tmp2 - 7;

release(L);

Thread 1 Thread 2

Same lock⇒ Mutuallyexclusive critical sections

Avoiding Data Races

int balance = 10;Thread 1 Thread 2

synchronized (L) {

int tmp1 = balance;

balance = tmp1 + 5;

synchronized (L) {

int tmp2 = balance;

balance = tmp2 - 7;

(Java syntax)

Outline

1. Introduction

Eraser: Data Race Detection

� Basic idea: Look for ”unprotected” accesses toshared memory

� Assumption: All accesses to a shared memorylocation v should happen while holding the samelock L

→ Consistent locking discipline

� Dynamic analysis that monitors all lockacquisitions, lock releases, and accesses ofshared memory locations

Lockset Algorithm (Simple Form)

� Let locksHeld(t) be the set of locksheld by thread t

� For each shared memory location v,initialize C(v) to the set of all locks

� On each access to v by thread t� Set C(v) := C(v) ∩ locksHeld(t)

� If C(v) = ∅, issue a warning

Lockset Algorithm (Simple Form)

� Let locksHeld(t) be the set of locksheld by thread t

� For each shared memory location v,initialize C(v) to the set of all locks

� On each access to v by thread t� Set C(v) := C(v) ∩ locksHeld(t)

� If C(v) = ∅, issue a warning

Lockset

Lockset refinement

Simple Lockset is Too Strict

Simple lockset algorithm produces falsepositives for� variables initialized without locks held

� read-shared data read without locks held

� read-write locking mechanisms(producer-consumer style)

Refining the Lockset Algorithm

Virgin

ExclusiveShared-modified

Shared

� Keep state of each shared memory location� Issue warnings only in the Shared-modified

writeread/write by1st thread

write by2nd thread

read by2nd thread

Summary: Eraser

� Dynamic analysis to detect data races

� Assumes consistent locking discipline

� Limitations� May report false positives when locks are

acquired inconsistently but correctly

� May miss data races because it does notconsider all possible interleavings

Outline

1. Introduction

Thread Safety

� Popular way to encapsulate the challenges ofconcurrent programming: Thread-safe classes

� Class ensures correct synchronization

� Clients can use instances as if they were alone

� Rest of program can treat implementation ofthread-safe class as a blackbox

Thread Safety (2)

“behaves correctly when accessedfrom multiple threads ... with noadditional synchronization ... (inthe) calling code”

“operations ... behave as if they occurin some serial order that is consistentwith the order of the method callsmade by each of the individualthreads”

StringBuffer API documentation, JDK 6

Example from JDK

StringBuffer b = new StringBuffer()

b.append("a")

b.append("b")

b.append("c")

Thread 1 Thread 2

Example from JDK

b.append("a")

b.append("b")

b.append("c")

Thread 1 Thread 2

Quiz: What can be the content of b ifStringBuffer is thread-safe?

Example from JDK

b.append("a")

b.append("b")

b.append("c")

Thread 1 Thread 2

"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7

Example from JDK

b.append("a")

b.append("b")

b.append("c")

Thread 1 Thread 2

"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7

Example from JDK

b.append("a")

b.append("b")

b.append("c")

Thread 1 Thread 2

"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7

Example from JDK

b.append("a")

b.append("b")

b.append("c")

Thread 1 Thread 2

"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7

Testing Thread-Safe Classes

� Correctness of program relies on thread safety ofspecific classes

� But: What if the class is actually not thread-safe?

� ConTeGe = Concurrent Test Generator

� Creates multi-threaded unit tests

� Detects thread safety violations by comparingconcurrent behavior against linearizations

Example Bug from JDK

b.append("abc")

b.insert(1, b) b.deleteCharAt(1)

Thread 1 Thread 2

IndexOutOfBoundsException

Confirmed as bug: Issue #7100996

ConTeGe

Classundertest(CUT)

Execute

Thread safetyoracle

Generate aconcurrent test

ConTeGe

Classundertest(CUT)

Execute

Thread safetyoracle

ConTeGe

Classundertest(CUT)

Execute

Thread safetyoracle

Generating Concurrent Tests

b.append("abc")

Thread 1 Thread 2

Example:

b.append("abc")

Thread 1 Thread 2

Sequential prefix:

Create and set upCUT instance

Example:

b.append("abc")

Thread 1 Thread 2

Concurrent suffixes:

Use shared CUTinstance

Example:

Test Generation Algorithm

1. Create prefix� Instantiate CUT

� Call methods

2. Create suffixes for prefix� Call methods on shared CUT instance

3. Prefix + two suffixes = test

Selection of methods similar tofeedback-directed test generation

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

Randomlyselect aconstructor

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

After adding a call:Execute

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

Randomlyselect amethod

b.append(/* String */)

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

b.append(/* String */)

Arguments:a) Take available objectb) Call method returning

required typec) Random value

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

Creating a Prefix

� Call methods

b.append("abc")

Thread 1 Thread 2

Creating Suffixes

2. Create suffixesfor prefix

� Call methods onshared CUT instance

Creating Suffixes

b.append("abc")

b.insert(1, b)

Randomlyselect amethod

b.insert(/* int */, /* CharSequence */)

Creating Suffixes

b.append("abc")

b.insert(1, b)b.insert(/* int */, /* CharSequence */)

Creating Suffixes

b.append("abc")

b.insert(1, b)b.insert(-5, b)

Creating Suffixes

b.append("abc")

b.insert(1, b)b.insert(-5, b)

Creating Suffixes

b.append("abc")

b.insert(1, b)b.insert(/* int */, /* CharSequence */)

Creating Suffixes

b.append("abc")

b.insert(1, b)

Creating Suffixes

b.append("abc")

b.insert(1, b)

Creating Suffixes

b.append("abc")

Creating Suffixes

b.append("abc")

Creating Suffixes

b.append("abc")

Creating a Test

3. Prefix + two suffixes = test

b.append("abc")

Thread 1 Thread 2

Spawn new threadfor each suffix

Approach

Classundertest(CUT)

Execute

Thread safetyoracle

Approach

Classundertest(CUT)

Execute

Thread safetyoracle

Thread Safety Oracle

Does the test executionexpose a thread safetyviolation?

� Focus on exceptionsand deadlocks

� Compare concurrentexecution tolinearizations

Linearizations

� Put all calls into one thread� Preserve order of calls within a thread

Linearizations

The Oracle

Exception ordeadlock?

Execute concurrently

No Thread safetyviolation

Samefailure?

Execute linearization All linearizationschecked

Example

b.append("abc")

StringBuffer b = ..

b.append("abc")

b.insert(1, b)

b.deleteCharAt(1) 3

Thread 1 Thread 2

StringBuffer b = ..

b.append("abc")

b.deleteCharAt(1)

b.insert(1, b) 3

Example

b.append("abc")

StringBuffer b = ..

b.append("abc")

b.insert(1, b)

b.deleteCharAt(1) 3

Thread 1 Thread 2

StringBuffer b = ..

b.append("abc")

b.deleteCharAt(1)

b.insert(1, b) 3

!Thread safety violation

Properties of the Oracle

Sound but incomplete *

� All reported violations are real� Cannot guarantee thread safety

Independent of bug type� Data races� Atomicity violations� Deadlocks

* with respect to incorrectness

Implementation & Results

� Implemented for Java classes

� Applied to popular thread-safe classesfrom JDK, Apache libraries, etc.

� Found 15 concurrency bugs, includingpreviously unknown problems in JDK

� Takes between several seconds andseveral hours (worst-case: 19 hours)� Recent master thesis, published at ICSE’17:

Worst-case time reduced to several minutes

Outline

1. Introduction

Scheduling Non-Determinism

� A single program executed with a single inputmay have many different interleavings

� Scheduler decides interleavingsnon-deterministically

� Some interleavings may expose bugs, othersexecute correctly (”Heisenbugs”)

� Challenge: How to explore different interleavings?How to detect buggy interleavings?

CHESS in a Nutshell

� A user mode scheduler that controlsall scheduling non-determinism

� Guarantees:� Every program run takes a new thread

interleaving� Can reproduce the interleaving for every run

� Systematic but non-exhaustiveexploration of the set of possibleinterleavings

Tree of Interleavings

� Search space of possibleinterleavings: Represent as a tree

� Node = points of scheduling decision

� Edge = decisions taken

� Each path = one possible schedule

Example

// bank account

int balance = 10;

// deposit money

int tmp1 = balance;

balance = tmp1 + 5;

// withdraw money

int tmp2 = balance;

balance = tmp2 - 7;

Thread 1 Thread 2

State Space Explosion

Thread 1:

instr. 1instr. 2...instr. k

n threads

k instructions

� Number ofinterleavings: O(nn·k)

� Exponential in both n

� Typically: n < 10,k > 100

� Exploring allinterleavings does notscale to largeprograms (i.e., large k)

Thread 2:

instr. 1instr. 2...instr. k

Preemption Bounding

� Limit exploration to schedules with a smallnumber c of preemptions� Preemption = Context switches forced by the

scheduler

� Number of schedules: O((n2 · k)c · n!)� Exponential in c and n, but not in k

� Based on empirical observation: Mostconcurrency bugs can be triggered with few (< 2)interleavings

Implementation and Results

� Implemented via binary instrumentation

� Applied to eight mid-size and large systems (upto 175K lines of code),

� Found a total of 27 bugs

� Major benefit over stress testing: Once a failure isdetected, can easily reproduce and debug it

Summary

� Concurrent programming is inevitable

� Writing correct concurrent programsis hard

� Techniques to detect concurrencybugs� Dynamic data race detection� Test generation and thread safety checking� Systematic exploration of interleavings

Testing Concurrent Programs Program Testing and Analysis€¦ · int tmp1 = balance; balance = tmp1...

Documents

Transcript of Testing Concurrent Programs Program Testing and Analysis€¦ · int tmp1 = balance; balance = tmp1...

COMMITTEE ROLL CALL - MAX Transit · Balance @ Balance @ Balance @ Balance @ Balance @ Balance @ 1/31/2018 12/31/2017 11/30/2017 10/31/2017 9/30/2017 8/31/2017 CASH ACCOUNTS: Money

Register Allocation for Irregular Architecturesdkoes/research/irregular.pdf · short a,b,c; r0 r1 r2 … r30 r31 Good register allocation ... EOR#3,tmp1 MOVEQ#32,tmp2or MOVEA #32,tmp2

Acid base balance + fluid balance

tableau competition2 P V tmp1 · tableau_competition2_P_V_tmp1.pdf Author: CHRISTOPHE Created Date: 1/30/2019 9:14:21 AM ...

Megawin MPC89E52 Data Sheet - Keil · TMP2 TMP1 ALU Stack Pointer Timer0/1 Timer2 UART PSW WDT Port1 Latch Port3 Latch Port4Latch Port1Driver Port3Driver Port4 Driver Control Unit

Dynamic Balance file Static Balance file Summary Balance files.

Plexus Balance | Product Information Sheet Plexus Balance · Balance is for individuals interested in weight management and a healthier lifestyle.* Block and Balance.* Plexus Balance

8-bit micro-controlleru.dianyuan.com/bbs/u/68/1242581218897698.pdf · TMP2 TMP1 ALU Stack Pointer Timer0 Timer1 UART PSW WDT Port1 Latch Port3 Latch Port1 Driver Port3 Driver Control

Copyright 2016 Cengage Learning. All Rights Reserved. May not …mawjj.people.ust.hk/math1013_course/Sec3-9.pdf · 2017-10-18 · Title: C:\Users\madli\Dropbox\m1013\tmp1.xps Author:

ETM 555 1 #include int main(int argc, char *argv[]) { long long tcKimlikNoNumber ; long long d[10] ; long long tmp1 = 0 ; long long tmp = 0 ; long long.

83C453/87C453 CMOS single-chip 8-bit microcontroller · tmp2 tmp1 alu timing and control instruction register pd oscillator psw port 1 latch port 3 latch port 1 drivers port 3 drivers

Herding Dogs and Balance What is balance?. Herding Dogs and Balance What is balance? Balance is an overall harmonious symmetry which a dog exhibits. A.

TRAFFIC MANAGEMENT PROTOCOL...Form TMP1 – Request for approval at preliminary design stage and request for approval at detailed design stage 4.0 Temporary Traffic Management request

POWER AMPLIFIER XP 600/1000 - ESpecarchive.espec.ws/files/PHONIC XP600_1000 .pdf · tmp1 a a power amplifier pcb phonic corporation xp 600/1000 service manual page 6 power amplifier

Acid Base Balance Fluid Balance

Balance BALANCE BEAM - TANDEM WALK

MEGAWIN - ic114.comproduct.ic114.com/PDF/M/MPC89x51A_A10.pdfTMP2 TMP1 ALU Stack Pointer Timer0/1 Timer2 UART PSW WDT Port1 Latch Port3 Latch Port4 Latch Port1 Driver Port3 Driver Port4

ABEL-HDL Primer€¦ · tmp1 node [istype 'com']; Created by J. Van der Spiegel, Sept. 26, 1997; Updated August 13, 1999 7 ... LARGE = B & C; D = [D3, D2,

T-M32 / T-M32 ASME · TMP1 TMP2 TMP3 MODE DIRE OUT D-PRT MST ON Keep the clearances. OUTDOOR INSTALLATION 1. Follow all local codes, or in the absence of local codes, follow the most

Balance of payments and Exchange Rates. Balance of payments & exchange rates The balance of payments account Example of UK balance of payments The balance.