Testing Concurrent Programs Program Testing and...
Transcript of Testing Concurrent Programs Program Testing and...
1
Program Testing and Analysis:
Testing Concurrent Programs
Dr. Michael Pradel
Software Lab, TU Darmstadt
2
Warm-up Quiz
var a = (0.1 + 0.2) + 0.3;var b = 0.1 + (0.2 + 0.3);console.log(a === b);
What does the following code print?
Something elsefalsetrue
2
Warm-up Quiz
var a = (0.1 + 0.2) + 0.3;var b = 0.1 + (0.2 + 0.3);console.log(a === b);
What does the following code print?
Something elsefalsetrue
2
Warm-up Quiz
var a = (0.1 + 0.2) + 0.3;var b = 0.1 + (0.2 + 0.3);console.log(a === b);
What does the following code print?
Something elsefalsetrue
Floating point numbers are representedwith finite precision(not only in JavaScript)
2
Warm-up Quiz
var a = (0.1 + 0.2) + 0.3;var b = 0.1 + (0.2 + 0.3);console.log(a === b);
What does the following code print?
Something elsefalsetrue
0.30000000000000004(due to rounding)
3
Mid-term
� Difficulty ≈What to expect for finalexam
� Exam and results will be madeavailable
� Results will be send out this week� Some students: Very good results� Other students: Have some work to do
3
Mid-term
4
Outline
1. Introduction
2. Dynamic Data Race Detection
3. Testing Thread-Safe Classes
4. Exploring Interleavings
Mostly based on these papers:
� Eraser: A Dynamic Data Race Detector for MultithreadedPrograms, Savage et al., ACM TOCS, 1997
� Fully Automatic and Precise Detection of Thread SafetyViolations, Pradel and Gross, PLDI 2012
� Finding and Reproducing Heisenbugs in ConcurrentPrograms, Musuvathi et al., USENIX 2008
5
Why Bother with Concurrency?
� The free lunch provided by Moore’s law is over� CPU clock speeds stopped to increase around 2005� Instead, multi-core processors became mainstream� Need concurrent programs to make full use of the
hardware
� Many real-world problems are inherentlyconcurrent, e.g.,� Servers must handle multiple concurrent requests� Computations done on huge data often are
”embarrasingly parallel”
5
Why Bother with Concurrency?
� The free lunch provided by Moore’s law is over� CPU clock speeds stopped to increase around 2005� Instead, multi-core processors became mainstream� Need concurrent programs to make full use of the
hardware
� Many real-world problems are inherentlyconcurrent, e.g.,� Servers must handle multiple concurrent requests� Computations done on huge data often are
”embarrasingly parallel”
5
Why Bother with Concurrency?
� The free lunch provided by Moore’s law is over� CPU clock speeds stopped to increase around 2005� Instead, multi-core processors became mainstream� Need concurrent programs to make full use of the
hardware
� Many real-world problems are inherentlyconcurrent, e.g.,� Servers must handle multiple concurrent requests� Computations done on huge data often are
”embarrasingly parallel”
6
Concurrency Styles
� Message-passing� Popular for large-scale scientific computing, e.g.,
MPI (message-passing interface)� Used in actor concurrency model, e.g., popular in
Erlang and Scala� No shared memory (ideally), all communication via
messages
� Thread-based, shared memory� Multiple concurrently executing threads� All threads access the same shared memory� Synchronize via locks and barriers
6
Concurrency Styles
� Message-passing� Popular for large-scale scientific computing, e.g.,
MPI (message-passing interface)� Used in actor concurrency model, e.g., popular in
Erlang and Scala� No shared memory (ideally), all communication via
messages
� Thread-based, shared memory� Multiple concurrently executing threads� All threads access the same shared memory� Synchronize via locks and barriers
Focus of this lecture
7
Example
int a = 0, b = 0;
boolean r = false, t = false;
a = 1;
r = true;
t = r;
b = a;
Thread 1 Thread 2
What does this program mean?
→ Behavior depends on threadinterleaving
1
9
Sequential Consistency
Assumption made here:Programs execute under sequential consistency
� Program order is preserved: Each thread’sinstructions execute in the specified order
� Shared memory behaves like a global array:Reads and writes are done immediately
� We assume sequential consistency for the rest ofthe lecture
� Many real-world platforms provide more complexsemantics (”memory models”)
10
What Can Go Wrong?
Common source of errors: Data races
� Two accesses to the same shared memorylocation
� At least one is a write
� Ordering of accesses is non-deterministic
11
Example
// bank account
int balance = 10;
// deposit money
int tmp1 = balance;
balance = tmp1 + 5;
// withdraw money
int tmp2 = balance;
balance = tmp2 - 7;
Thread 1 Thread 2
11
Example
// bank account
int balance = 10;
// deposit money
int tmp1 = balance;
balance = tmp1 + 5;
// withdraw money
int tmp2 = balance;
balance = tmp2 - 7;
Thread 1 Thread 2
Sharedmemorylocation
Read
Write
11
Example
// bank account
int balance = 10;
// deposit money
int tmp1 = balance;
balance = tmp1 + 5;
// withdraw money
int tmp2 = balance;
balance = tmp2 - 7;
Thread 1 Thread 2
3 races
11
Example
// bank account
int balance = 10;
// deposit money
int tmp1 = balance;
balance = tmp1 + 5;
// withdraw money
int tmp2 = balance;
balance = tmp2 - 7;
Thread 1 Thread 2
Quiz: What values can balance
have after executing this code?
11
Example
// bank account
int balance = 10;
// deposit money
int tmp1 = balance;
balance = tmp1 + 5;
// withdraw money
int tmp2 = balance;
balance = tmp2 - 7;
Thread 1 Thread 2
Possible outcomes:balance may be 3, 8, and 15
But: Only 8 is correct
12
Avoiding Data Races
Use locks to ensure that accesses toshared memory do not interfere
int balance = 10;
acquire(L);
int tmp1 = balance;
balance = tmp1 + 5;
release(L);
acquire(L);
int tmp2 = balance;
balance = tmp2 - 7;
release(L);
Thread 1 Thread 2
12
Avoiding Data Races
Use locks to ensure that accesses toshared memory do not interfere
int balance = 10;
acquire(L);
int tmp1 = balance;
balance = tmp1 + 5;
release(L);
acquire(L);
int tmp2 = balance;
balance = tmp2 - 7;
release(L);
Thread 1 Thread 2
Same lock⇒ Mutuallyexclusive critical sections
12
Avoiding Data Races
Use locks to ensure that accesses toshared memory do not interfere
int balance = 10;Thread 1 Thread 2
synchronized (L) {
int tmp1 = balance;
balance = tmp1 + 5;
}
synchronized (L) {
int tmp2 = balance;
balance = tmp2 - 7;
}
(Java syntax)
13
Outline
1. Introduction
2. Dynamic Data Race Detection
3. Testing Thread-Safe Classes
4. Exploring Interleavings
Mostly based on these papers:
� Eraser: A Dynamic Data Race Detector for MultithreadedPrograms, Savage et al., ACM TOCS, 1997
� Fully Automatic and Precise Detection of Thread SafetyViolations, Pradel and Gross, PLDI 2012
� Finding and Reproducing Heisenbugs in ConcurrentPrograms, Musuvathi et al., USENIX 2008
14
Eraser: Data Race Detection
� Basic idea: Look for ”unprotected” accesses toshared memory
� Assumption: All accesses to a shared memorylocation v should happen while holding the samelock L
→ Consistent locking discipline
� Dynamic analysis that monitors all lockacquisitions, lock releases, and accesses foshared memory locations
15
Lockset Algorithm (Simple Form)
� Let locksHeld(t) be the set of locksheld by thread t
� For each shared memory location v,initialize C(v) to the set of all locks
� On each access to v by thread t� Set C(v) := C(v) ∩ locksHeld(t)
� If C(v) = ∅, issue a warning
15
Lockset Algorithm (Simple Form)
� Let locksHeld(t) be the set of locksheld by thread t
� For each shared memory location v,initialize C(v) to the set of all locks
� On each access to v by thread t� Set C(v) := C(v) ∩ locksHeld(t)
� If C(v) = ∅, issue a warning
Lockset
Lockset refinement
2
17
Simple Lockset is Too Strict
Simple lockset algorithm produces falsepositives for� variables initialized without locks held
� read-shared data read without locks held
� read-write locking mechanisms(producer-consumer style)
18
Refining the Lockset Algorithm
Virgin
ExclusiveShared-modified
Shared
� Keep state of each shared memory location� Issue warnings only in the Shared-modified
state
wrrd/wr by1st thread
wr by 2nd thread
rd by 2ndthread
rd
wr
19
Summary: Eraser
� Dynamic analysis to detect data races
� Assumes consistent locking discipline
� Limitations� May report false positives when locks are
acquired inconsistently but correctly
� May miss data races because it does notconsider all possible interleavings
20
Outline
1. Introduction
2. Dynamic Data Race Detection
3. Testing Thread-Safe Classes
4. Exploring Interleavings
Mostly based on these papers:
� Eraser: A Dynamic Data Race Detector for MultithreadedPrograms, Savage et al., ACM TOCS, 1997
� Fully Automatic and Precise Detection of Thread SafetyViolations, Pradel and Gross, PLDI 2012
� Finding and Reproducing Heisenbugs in ConcurrentPrograms, Musuvathi et al., USENIX 2008
21
Thread Safety
� Popular way to encapsulate the challenges ofconcurrent programming: Thread-safe classes
� Class ensures correct synchronization
� Clients can use instances as if they were alone
� Rest of program can treat implementation ofthread-safe class as a blackbox
22
Thread Safety (2)
“behaves correctly when accessedfrom multiple threads ... with noadditional synchronization ... (inthe) calling code” page 18
22
Thread Safety (2)
“behaves correctly when accessedfrom multiple threads ... with noadditional synchronization ... (inthe) calling code” page 18
22
Thread Safety (2)
“behaves correctly when accessedfrom multiple threads ... with noadditional synchronization ... (inthe) calling code” page 18
22
Thread Safety (2)
“behaves correctly when accessedfrom multiple threads ... with noadditional synchronization ... (inthe) calling code”
“operations ... behave as if they occurin some serial order that is consistentwith the order of the method callsmade by each of the individualthreads”
page 18
StringBuffer API documentation, JDK 6
22
Thread Safety (2)
“behaves correctly when accessedfrom multiple threads ... with noadditional synchronization ... (inthe) calling code”
“operations ... behave as if they occurin some serial order that is consistentwith the order of the method callsmade by each of the individualthreads”
page 18
StringBuffer API documentation, JDK 6
23
Example from JDK
StringBuffer b = new StringBuffer()
b.append("a")
b.append("b")
b.append("c")
Thread 1 Thread 2
23
Example from JDK
StringBuffer b = new StringBuffer()
b.append("a")
b.append("b")
b.append("c")
Thread 1 Thread 2
Quiz: What can be the content of b ifStringBuffer is thread-safe?
23
Example from JDK
StringBuffer b = new StringBuffer()
b.append("a")
b.append("b")
b.append("c")
Thread 1 Thread 2
"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7
23
Example from JDK
StringBuffer b = new StringBuffer()
b.append("a")
b.append("b")
b.append("c")
Thread 1 Thread 2
"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7
23
Example from JDK
StringBuffer b = new StringBuffer()
b.append("a")
b.append("b")
b.append("c")
Thread 1 Thread 2
"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7
23
Example from JDK
StringBuffer b = new StringBuffer()
b.append("a")
b.append("b")
b.append("c")
Thread 1 Thread 2
"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7
23
Example from JDK
StringBuffer b = new StringBuffer()
b.append("a")
b.append("b")
b.append("c")
Thread 1 Thread 2
"abc" 3 "cab" 3 "acb" 3 "ac" 7 "bac" 7
24
Testing Thread-Safe Classes
� Correctness of program relies on thread safety ofspecific classes
� But: What if the class is actually not thread-safe?
� ConTeGe = Concurrent Test Generator
� Creates multi-threaded unit tests
� Detects thread safety violations by comparingconcurrent behavior against linearizations
25
Example Bug from JDK
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
25
Example Bug from JDK
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
IndexOutOfBoundsException
Confirmed as bug: Issue #7100996
!
26
ConTeGe
Bug
Classundertest(CUT)
Execute
Thread safetyoracle
Generate aconcurrent test
26
ConTeGe
Bug
Classundertest(CUT)
Execute
Thread safetyoracle
Generate aconcurrent test
26
ConTeGe
Bug
Classundertest(CUT)
Execute
Thread safetyoracle
Generate aconcurrent test
27
Generating Concurrent Tests
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Example:
27
Generating Concurrent Tests
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Sequential prefix:
Create and set upCUT instance
Example:
27
Generating Concurrent Tests
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Concurrent suffixes:
Use shared CUTinstance
Example:
28
Test Generation Algorithm
1. Create prefix� Instantiate CUT
� Call methods
2. Create suffixes for prefix� Call methods on shared CUT instance
3. Prefix + two suffixes = test
Selection of methods similar tofeedback-directed test generation
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Randomlyselect aconstructor
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Randomlyselect aconstructor
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
After adding a call:Execute
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
After adding a call:Execute
3
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Randomlyselect amethod
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Randomlyselect amethod
b.append(/* String */)
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
b.append(/* String */)
Arguments:a) Take available objectb) Call method returning
required typec) Random value
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Arguments:a) Take available objectb) Call method returning
required typec) Random value
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
After adding a call:Execute
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
After adding a call:Execute
3
29
Creating a Prefix
1. Create prefix� Instantiate CUT
� Call methods
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)
Randomlyselect amethod
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)
Randomlyselect amethod
b.insert(/* int */, /* CharSequence */)
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)b.insert(/* int */, /* CharSequence */)
Arguments:a) Take available objectb) Call method returning
required typec) Random value
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)b.insert(-5, b)
Arguments:a) Take available objectb) Call method returning
required typec) Random value
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)b.insert(-5, b)
After adding a call:Execute
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)b.insert(-5, b)
After adding a call:Execute
!
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)b.insert(/* int */, /* CharSequence */)
Arguments:a) Take available objectb) Call method returning
required typec) Random value
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)
Arguments:a) Take available objectb) Call method returning
required typec) Random value
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)
After adding a call:Execute
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)
After adding a call:Execute
3
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b)
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
After adding a call:Execute
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
After adding a call:Execute
3
30
Creating Suffixes
2. Create suffixesfor prefix
� Call methods onshared CUT instance
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
31
Creating a Test
3. Prefix + two suffixes = test
31
Creating a Test
3. Prefix + two suffixes = test
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
31
Creating a Test
3. Prefix + two suffixes = test
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
Spawn new threadfor each suffix
32
Approach
Bug
Classundertest(CUT)
Execute
Thread safetyoracle
Generate aconcurrent test
32
Approach
Bug
Classundertest(CUT)
Execute
Thread safetyoracle
Generate aconcurrent test
33
Thread Safety Oracle
Does the test executionexpose a thread safetyviolation?
� Focus on exceptionsand deadlocks
� Compare concurrentexecution tolinearizations
34
Assumptions
Concurrency-only crashes are undesired
� Matches definition of thread safety
Control over all input to tests
� Sequential execution: Deterministic
35
Linearizations
� Put all calls into one thread� Preserve order of calls within a thread
35
Linearizations
� Put all calls into one thread� Preserve order of calls within a thread
21 3
35
Linearizations
213
2
13
21
3
� Put all calls into one thread� Preserve order of calls within a thread
21 3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
3
3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
Execute concurrently
3
3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
Exception ordeadlock? 3
3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
Execute linearization
3
3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
Samefailure?
3
3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
Execute linearization
3
3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
Samefailure?
3
3
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
3
3
All linearizationschecked
36
The Oracle
Exception ordeadlock?
Execute concurrently
No
Yes
Yes
No Thread safetyviolation
Samefailure?
Execute linearization All linearizationschecked
Thread safetyviolation
3
3
37
Example
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
37
Example
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
Thread 1 Thread 2
!
37
Example
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
StringBuffer b = ..
b.append("abc")
b.insert(1, b)
b.deleteCharAt(1) 3
Thread 1 Thread 2
!
37
Example
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
StringBuffer b = ..
b.append("abc")
b.insert(1, b)
b.deleteCharAt(1) 3
Thread 1 Thread 2
StringBuffer b = ..
b.append("abc")
b.deleteCharAt(1)
b.insert(1, b) 3
!
37
Example
StringBuffer b = new StringBuffer()
b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
StringBuffer b = ..
b.append("abc")
b.insert(1, b)
b.deleteCharAt(1) 3
Thread 1 Thread 2
StringBuffer b = ..
b.append("abc")
b.deleteCharAt(1)
b.insert(1, b) 3
!Thread safety violation
38
Properties of the Oracle
Sound but incomplete *
� All reported violations are real� Cannot guarantee thread safety
Independent of bug type� Data races� Atomicity violations� Deadlocks
* with respect to incorrectness
39
Implementation & Results
� Implemented for Java classes
� Applied to popular thread-safe classesfrom JDK, Apache libraries, etc.
� Found 15 concurrency bugs, includingpreviously unknown problems in JDK
� Takes between several seconds andseveral hours (worst case: 19 hours)
40
Open Challenges
� How to generate tests that are likely to triggerbugs? (Currently: random decisions)
� Static analysis to find potential bugs; focus on thoseduring test generation
� Use feedback from test execution to steer testgeneration towards not yet explored behavior
� How to generate tests for larger pieces ofconcurrent software, e.g., entire libraries orprograms?
40
Open Challenges
� How to generate tests that are likely to triggerbugs? (Currently: random decisions)
� Static analysis to find potential bugs; focus on thoseduring test generation
� Use feedback from test execution to steer testgeneration towards not yet explored behavior
� How to generate tests for larger pieces ofconcurrent software, e.g., entire libraries orprograms?
Hint: Opportunities for master theses
41
Outline
1. Introduction
2. Dynamic Data Race Detection
3. Testing Thread-Safe Classes
4. Exploring Interleavings
Mostly based on these papers:
� Eraser: A Dynamic Data Race Detector for MultithreadedPrograms, Savage et al., ACM TOCS, 1997
� Fully Automatic and Precise Detection of Thread SafetyViolations, Pradel and Gross, PLDI 2012
� Finding and Reproducing Heisenbugs in ConcurrentPrograms, Musuvathi et al., USENIX 2008
42
Scheduling Non-Determinism
� A single program executed with a single inputmay have many different interleavings
� Scheduler decides interleavingsnon-deterministically
� Some interleavings may expose bugs, othersexecute correctly (”Heisenbugs”)
� Challenge: How to explore different interleavings?How to detect buggy interleavings?
43
CHESS in a Nutshell
� A user mode scheduler that controlsall scheduling non-determinism
� Guarantees:� Every program run takes a new thread
interleaving� Can reproduce the interleaving for every run
� Systematic but non-exhaustiveexploration of the set of possibleinterleavings
44
Tree of Interleavings
� Search space of possibleinterleavings: Represent as a tree
� Node = points of scheduling decision
� Edge = decisions taken
� Each path = one possible schedule
45
Example
// bank account
int balance = 10;
// deposit money
int tmp1 = balance;
balance = tmp1 + 5;
// withdraw money
int tmp2 = balance;
balance = tmp2 - 7;
Thread 1 Thread 2
3
47
State Space Explosion
Thread 1:
instr. 1instr. 2...instr. k
n threads
k instructions
� Number ofinterleavings: O(nn·k)
� Exponential in both n
and k
� Typically: n < 10,k > 100
� Exploring allinterleavings does notscale to largeprograms (i.e., large k)
Thread 2:
instr. 1instr. 2...instr. k
48
Preemption Bounding
� Limit exploration to schedules with a smallnumber c of preemptions� Preemption = Context switches forced by the
scheduler
� Number of schedules: O((n2 · k)c · n!)� Exponential in c and n, but not in k
� Based on empirical observation: Mostconcurrency bugs can be triggered with few (< 2)interleavings
49
Implementation and Results
� Implemented via binary instrumentation
� Applied to eight mid-size and large systems (upto 175K lines of code),
� Found a total of 27 bugs
� Major benefit over stress testing: Once a failure isdetected, can easily reproduce and debug it
50
Summary
� Concurrent programming is inevitable
� Writing correct concurrent programsis hard
� Techniques to detect concurrencybugs� Dynamic data race detection� Test generation and thread safety checking� Systematic exploration of interleavings