Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of...
-
Upload
allen-mcgee -
Category
Documents
-
view
213 -
download
1
Transcript of Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of...
![Page 1: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/1.jpg)
1
Cooperative Concurrency Bug Isolation
Guoliang Jin, Aditya Thakur, Ben Liblit, Shan LuUniversity of Wisconsin–Madison
Instrumentation and Sampling Strategies
for
![Page 2: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/2.jpg)
2
Cooperative Concurrency Bug Isolation
• They are synchronization mistakes in multi-threaded programs.
• Several types:– Atomicity violation– Data race– Deadlock, etc.
read(x)
read(x)
write(x)
thread 1 thread 2
JL
write(x)
read(x)
thread 1 thread 2
J?J?
![Page 3: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/3.jpg)
3
Concurrency bugs are common in the fields
• Developers are poor at parallel programming• Interleaving testing is inefficient• Applications with concurrency bugs shipped to
the users
�ƒ€‚�
![Page 4: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/4.jpg)
4
Concurrency bug lead to failures in the field
• Disasters in the past– Therac-25, Northeastern Blackout 2003
• More threats in multi-core era
‚
![Page 5: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/5.jpg)
5
Failure diagnosis is critical
![Page 6: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/6.jpg)
6
L
Concurrency Bug Failure Example
Concurrency Bug from Apache HTTP Server
![Page 7: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/7.jpg)
7
…memcpy(&buf[idx], s, strlen(s));
…log_writer() {
…}…
thread 1
J
Concurrency Bug Failure Example
Concurrency Bug from Apache HTTP Server
…temp = idx;idx = temp + strlen(s);
idx
thread 2
…return SUCCESS;
…memcpy(&buf[idx], s, strlen(s));
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);…return SUCCESS;
![Page 8: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/8.jpg)
8
…return SUCCESS;
…memcpy(&buf[idx], s, strlen(s));
…memcpy(&buf[idx], s, strlen(s));
…log_writer() {
…}…
thread 1
L
Concurrency Bug Failure Example
Concurrency Bug from Apache HTTP Server
…temp = idx;idx = temp + strlen(s);
idx
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);
![Page 9: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/9.jpg)
9
• The failure is non-deterministic and rare– Programmers have trouble to repeat the failure
• The root cause involves more than one thread
Diagnosing Concurrency Bug Failure is Challenging
![Page 10: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/10.jpg)
10
Existing work and their limitations
• Failure replay– High runtime overhead– Developers need to manually locate faults
• Run-time bug detection– (mostly) High runtime overhead– Not guided by the failure• Many false positives How to achieve
low-overhead & accurate
failure diagnosis?
![Page 11: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/11.jpg)
11
Predicates
Our work: CCI
�ƒƒ€‚�
Program
SourceCompiler
Counts& J/L
StatisticalDebugging
Predictors
Sampler
• Goal: diagnosing production run concurrency bug failures• Major components:– predicates instrumentor– sampler– statistical debugging
True in most failure runs, false in most correct runs.
![Page 12: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/12.jpg)
12
CCI Overview• Three different types of predicates.• Each predicate has its supporting
sampling strategy.• Same statistical debugging as in CBI.• Experiments show CCI is effective in
diagnosing concurrency failures.
Capability
Ove
rhea
d
FunRe
Havoc
Prev
![Page 13: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/13.jpg)
13
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
Outline
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
![Page 14: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/14.jpg)
14
CCI-Prev Intuition
read(x)
read(x)
write(x)
J L
thread 1 thread 2
read(x)
read(x)
write(x)
thread 1 thread 2
read(x)
write(x)
J L
thread 1 thread 2
read(x)
write(x)
thread 1 thread 2
Atomicity Violation Data Race
Just record which thread accessed last time.
read(x) write(x)
read(x)
read(x)
read(x)
write(x) read(x)
![Page 15: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/15.jpg)
15
CCI-Prev PredicateIt tracks whether two successive accesses to
a shared memory location were by two distinct threads or were by the same thread.
Capability
Ove
rhea
d Prev
![Page 16: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/16.jpg)
16
…memcpy(&buf[idx], s, strlen(s));
…memcpy(&buf[idx], s, strlen(s));
…log_writer() {
…}…
thread 1
J
CCI-Prev Predicate on the Correct Run
Concurrency Bug from Apache HTTP Server
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);…return SUCCESS;
I
I
Predicate J L…
remoteI 0 0
localI 0 0
…
Predicate J L…
remoteI 0 0
localI 1 0
…
Predicate J L…
remoteI 0 0
localI 2 0
…
![Page 17: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/17.jpg)
17
…memcpy(&buf[idx], s, strlen(s));
…memcpy(&buf[idx], s, strlen(s));
…return SUCCESS;
…log_writer() {
…}…
thread 1
L
CCI-Prev Predicate on the Failure Run
Concurrency Bug from Apache HTTP Server
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);
I
I
Predicate J L…
remoteI 0 0
localI 2 0
…
Predicate J L…
remoteI 0 0
localI 2 1
…
Predicate J L…
remoteI 0 1
localI 2 1
…
Predicate J L…
remoteI 0 1
localI 2 1
…
Predicate J L…
remoteI 0 1
localI 2 1
…
![Page 18: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/18.jpg)
18
…memcpy(&buf[idx], s, strlen(s));
…
…log_writer() {
…}…
thread 1
L
CCI-Prev Predicate Instrumentation
Concurrency Bug from Apache HTTP Server
temp = idx;
idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {…}…
Predicate J L…
remoteI 0 0
localI 2 1
…
Predicate J L…
remoteI 0 1
localI 2 1
…
Iunlock(glock);
remote = test_and_insert(& idx, curTid);record(I, remote);
lock(glock);a global hash table
address ThreadID
… …
& idx 2
… …
address ThreadID
… …
& idx 1
… …
address ThreadID
… …
& idx 1
… …
![Page 19: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/19.jpg)
19
…memcpy(&buf[idx], s, strlen(s));
…memcpy(&buf[idx], s, strlen(s));
…return SUCCESS;
…log_writer() {
…}…
thread 1
CCI-Prev Sampling Strategy
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);
Does traditional sampling work? NO.
• Thread-coordinated• Bursty
I
![Page 20: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/20.jpg)
20
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
Outline
![Page 21: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/21.jpg)
21
…memcpy(&buf[idx], s, strlen(s));
CCI-Havoc Intuition
Just record what value was observed during last access.
…memcpy(&buf[idx], s, strlen(s));
…return SUCCESS;
…log_writer() {
…}…
thread 1
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);
I
![Page 22: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/22.jpg)
22
CCI-Havoc PredicateIt tracks whether the value of a given shared location changes between two consecutive accesses by one thread.
Capability
Ove
rhea
d Prev
Havoc
Only uses thread local information
![Page 23: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/23.jpg)
23
…memcpy(&buf[idx], s, strlen(s));
…memcpy(&buf[idx], s, strlen(s));
…log_writer() {
…}…
thread 1
J
CCI-Havoc Predicate on the Correct Run
Concurrency Bug from Apache HTTP Server
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);…return SUCCESS;
I
I
Predicate J L…
unchangedI 0 0
changedI 0 0
…
Predicate J L…
unchangedI 1 0
changedI 0 0
…
Predicate J L…
unchangedI 2 0
changedI 0 0
…
![Page 24: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/24.jpg)
24
…memcpy(&buf[idx], s, strlen(s));
…memcpy(&buf[idx], s, strlen(s));
…return SUCCESS;
…log_writer() {
…}…
thread 1
L
CCI-Havoc Predicate on the Failure Run
Concurrency Bug from Apache HTTP Server
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);
I
I
Predicate J L…
unchangedI 2 0
changedI 0 0
…
Predicate J L…
unchangedI 2 1
changedI 0 0
…
Predicate J L…
unchangedI 2 1
changedI 0 1
…
Predicate J L…
unchangedI 2 1
changedI 0 1
…
Predicate J L…
unchangedI 2 1
changedI 0 1
…
![Page 25: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/25.jpg)
25
…memcpy(&buf[idx], s, strlen(s));
…log_writer() {
…}…
thread 1
L
CCI-Havoc Predicate Instrumentation
Concurrency Bug from Apache HTTP Server
… temp = idx;
idx = temp + strlen(s);
thread 2
…return SUCCESS;
Predicate J L…
unchangedI 2 1
changedI 0 0
…
Predicate J L…
unchangedI 2 1
changedI 0 1
…
…log_writer() {…}…
I
insert (& idx, temp);
changed = test(& idx, temp);record(I, changed);
hash table forthread1
address value
… …
& idx idx
… …
address value
… …
& idx idx+len2
… …
![Page 26: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/26.jpg)
26
…memcpy(&buf[idx], s, strlen(s));
…return SUCCESS;
…log_writer() {
…}…
thread 1
CCI-Havoc Sampling Strategy
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);
• Bursty• Thread-independent
…memcpy(&buf[idx], s, strlen(s));
![Page 27: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/27.jpg)
27
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
Outline
![Page 28: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/28.jpg)
28
CCI-FunRe PredicateIt tracks whether the execution of one function overlaps with the execution of the same function from a different thread.
Capability
Ove
rhea
d Prev
HavocFunRe
![Page 29: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/29.jpg)
CCI-FunRe Predicate Examplethread 1 thread 2
L
thread 1 thread 2
J
…log_writer() {…return SUCCESS;}… …
log_writer() {…return SUCCESS;}…
…log_writer() {…
return SUCCESS;}…
…log_writer() {…return SUCCESS;}…
Predicate J L…
NonReentlog_writer 2 1
Reentlog_writer 0 1
…
Predicate J L…
NonReentlog_writer 2 1
Reentlog_writer 0 1
… 29
![Page 30: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/30.jpg)
30
…log_writer() {
oldCount = atomic_inc(Count); record(“log_writer”, oldCount);
…
atomic_dec(Count); return SUCCESS;}…
CCI-FunRe Predicate Instrumentationthread 1 thread 2
…log_writer() {
oldCount = atomic_inc(Count); record(“log_writer”, oldCount);
…
atomic_dec(Count); return SUCCESS;}…
L
Predicate J L…
NonReentlog_writer 2 0
Reentlog_writer 0 0
…
FuncName Counter
… …
log_writer 0
… …
FuncName Counter
… …
log_writer 1
… …
Predicate J L…
NonReentlog_writer 2 1
Reentlog_writer 0 0
…
FuncName Counter
… …
log_writer 2
… …
Predicate J L…
NonReentlog_writer 2 1
Reentlog_writer 0 1
…
Predicate J L…
NonReentlog_writer 2 1
Reentlog_writer 0 1
…
FuncName Counter
… …
log_writer 0
… …
![Page 31: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/31.jpg)
31
CCI-FunRe Sampling Strategy
L
thread 1 thread 2…log_writer() {
…
return SUCCESS;}…
Function execution accounting is not suitable for sampling, so this part is unconditional.
…log_writer() {
oldCount = atomic_inc(Count); record(“log_writer”, oldCount);
…
atomic_dec(Count); return SUCCESS;}…
FuncName Counter
… …
log_writer 0
… …
FuncName Counter
… …
log_writer 0
… …
FuncName Counter
… …
log_writer 0
… …
![Page 32: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/32.jpg)
32
CCI-FunRe Sampling Strategy
• Function execution accounting:–unconditional
• FunRe predicate recording:–thread-independent–non-bursty
![Page 33: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/33.jpg)
33
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
• Motivation• CCI Overview• CCI Predicates and Sampling Strategies – CCI-Prev and its sampling strategy – CCI-Havoc and its sampling strategy– CCI-FunRe and its sampling strategy
• Evaluation• Conclusion
Outline
![Page 34: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/34.jpg)
34
Experimental Evaluation
• Implementation– Static instrumentor based on the CBI framework
• Real world concurrency bug failure from:– Apache HTTP server, Cherokee– Mozilla-JS, PBZIP2– SPLASH-2: FFT, LU
• Parameter used– Roughly 1/100 sampling rate
![Page 35: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/35.jpg)
35
Failure Diagnosis Evaluation
• Methodology– Using concurrency bug failures occurred in real-world– Each app. runs 3000 times on a multi-core machine• Add random sleep to get some failure runs
– Sampling is enabled– Statistical debugging then return a list of predictors• Which predictor in the list can diagnose failure?
![Page 36: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/36.jpg)
36
Failure Diagnosis Results (with sampling)
Program CCI-Prev CCI-Havoc CCI-FunRe
Apache-1 top1 top1 top1Apache-2 top1 top1 Cherokee top2
FFT top1 LU top1
Mozilla-JS-1 top2 top1Mozilla-JS-2 top1 top1 top1Mozilla-JS-3 top2 top1 top1
PBZIP2 top1 top1
FunRe Havoc Prev
Capability
![Page 37: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/37.jpg)
37
Runtime OverheadPrev Havoc FunRe
No Sampling
Sampling No Sampling
Sampling No Sampling
Sampling
Apache-1 62.6% 27.4% 1.1%
Apache-2 8.4% 4.2% 0.2%
Cherokee 19.1% 2.1% 0.3%
FFT 169 % 33.5% 72.8%
LU 57857 % 1693 % 1682 %
Mozilla-JS 11311 % 7587 % 123 %
PBZIP2 0.2% 0.2% 0.3%
FunRe Havoc Prev
Overhead
Prev Havoc FunRe
No Sampling
Sampling No Sampling
Sampling No Sampling
Sampling
Apache-1 62.6% 1.9% 27.4% 2.8% 1.1% 1.8%
Apache-2 8.4% 0.5% 4.2% 0.4% 0.2% 0.2%
Cherokee 19.1% 0.3% 2.1% 0.0% 0.3% 0.4%
FFT 169 % 24.0% 33.5% 5.5% 72.8% 30.0%
LU 57857 % 949 % 1693 % 8.9% 1682 % 926 %
Mozilla-JS 11311 % 606 % 7587 % 356 % 123 % 97.0%
PBZIP2 0.2% 0.2% 0.2% 0.2% 0.3% 0.2%
![Page 38: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/38.jpg)
38
Conclusion• CCI is capable and suitable to
diagnose many production-run concurrency bug failures.
• Future predicates can leverage our effective sampling strategies.
• Experiments confirm design tradeoff.
Capability
Ove
rhea
d
Prev
Havoc
FunRe
![Page 39: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/39.jpg)
39
Questions about ?
Capability
Ove
rhea
d
Prev
Havoc
FunRe
CCI
![Page 40: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/40.jpg)
40
Questions about ?
Capability
Ove
rhea
d
Prev
Havoc
FunRe
CCI
![Page 41: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/41.jpg)
41
…memcpy(&buf[idx], s, strlen(s));
…memcpy(&buf[idx], s, strlen(s));
CBI on Concurrency Bug Failures
…return SUCCESS;
…log_writer() {
…}…
thread 1
LConcurrency Bug from Apache HTTP Server
…temp = idx;idx = temp + strlen(s);
thread 2
…return SUCCESS;
…log_writer() {
…}…
…temp = idx;idx = temp + strlen(s);
CBI does not work!
idx
To diagnose production-run concurrency bug failures, interleaving related events should be tracked!!!
![Page 42: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/42.jpg)
42
CCI-Prev Predicate Instrumentation with Sampling
if (gsample) {
} else {
temp = cnt;
lock(glock);
changed = test_and_insert(& cnt, curTid);
record(I, changed);
temp = cnt;
unlock(glock);
[[ gsample = true; iset = curTid; lLength=gLength=0;]]?}
![Page 43: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/43.jpg)
43
CCI-Prev Predicate Instrumentation with Sampling
if (gsample) {
} else {
temp = cnt;
lock(glock);
changed = test_and_insert(& cnt, curTid);
record(I, changed);
temp = cnt;
[[ gsample = true; iset = curTid; lLength=gLength=0;]]?
}
unlock(glock);
lLength++;
gLength++;
if (( iset == curTid && lLength > lMAX) || gLength > gMAX){ clear (); iset = unusedTid; gsample = false; }
record(stale ? P1 : P2, changed);
changed = test_and_insert(& cnt, curTid, &stale);
![Page 44: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/44.jpg)
44
CCI-Havoc Predicate Instrumentation with Sampling
record(stale ? P1 : P2, changed);
changed = test(& cnt, cnt, &stale);
if (sample) {
} else {
temp = cnt;
temp = cnt;
[[ sample = true; length=0;]]?
}
insert (& cnt, cnt);
if (length > lMAX) { clear (); sample = false;}
length++;
No global lock used!!!
![Page 45: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/45.jpg)
45
Failure Diagnosis Results (with sampling)
Program CBI CCI-Prev CCI-Havoc CCI-FunRe
Apache-1 top1 top1 top1Apache-2 top1 top1 Cherokee top2
FFT top1 LU top1
Mozilla-JS-1 top2 top1Mozilla-JS-2 top1 top1 top1Mozilla-JS-3 top2 top1 top1
PBZIP2 top1 top1
FunRe Havoc Prev
Capability
![Page 46: Cooperative Concurrency Bug Isolation Guoliang Jin, Aditya Thakur, Ben Liblit, Shan Lu University of Wisconsin–Madison Instrumentation and Sampling Strategies.](https://reader035.fdocuments.us/reader035/viewer/2022070403/56649f305503460f94c4ab9a/html5/thumbnails/46.jpg)
46
Failure diagnosis is critical