Poirot – A Concurrency Sleuth Shaz Qadeer Research in Software Engineering Microsoft Research.
-
Upload
marilyn-jenkins -
Category
Documents
-
view
233 -
download
4
Transcript of Poirot – A Concurrency Sleuth Shaz Qadeer Research in Software Engineering Microsoft Research.
Poirot – A Concurrency Sleuth
Shaz QadeerResearch in Software Engineering
Microsoft Research
Concurrent programming is difficult
…if (irp->Cancel) { IoCompleteIrp(irp);} else { IoSetCancelRoutine(irp, CancelRoutine); IoMarkIrpPending(irp);}…
…irp->Cancel = TRUE;fn = IoSetCancelRoutine(Irp, NULL);if (fn) { fn(irp);}…
void CancelRoutine(IRP *irp) { IoCompleteIrp(irp);}
Normal Cancellation
IO_REQUEST_PACKET *irp;
irp->Cancel = FALSE;irp->CancelRoutine = NULL;
…if (irp->Cancel) { IoCompleteIrp(irp);} else { IoSetCancelRoutine(irp, CancelRoutine); IoMarkIrpPending(irp);}…
…irp->Cancel = TRUE;fn = IoSetCancelRoutine(Irp, NULL);if (fn) { fn(irp);}…
void CancelRoutine(IRP *irp) { IoCompleteIrp(irp);}
Normal Cancellation
IO_REQUEST_PACKET *irp;
irp->Cancel = FALSE;irp->CancelRoutine = NULL;
…if (irp->Cancel) { IoCompleteIrp(irp);} else { IoSetCancelRoutine(irp, CancelRoutine); IoMarkIrpPending(irp);}…
…irp->Cancel = TRUE;fn = IoSetCancelRoutine(Irp, NULL);if (fn) { fn(irp);}…
void CancelRoutine(IRP *irp) { IoCompleteIrp(irp);}
Normal Cancellation
IO_REQUEST_PACKET *irp;
irp->Cancel = FALSE;irp->CancelRoutine = NULL;
…if (irp->Cancel) { IoCompleteIrp(irp);} else { IoSetCancelRoutine(irp, CancelRoutine); IoMarkIrpPending(irp);}…
…irp->Cancel = TRUE;fn = IoSetCancelRoutine(Irp, NULL);if (fn) { fn(irp);}…
void CancelRoutine(IRP *irp) { IoCompleteIrp(irp);}
Normal Cancellation
IO_REQUEST_PACKET *irp;
irp->Cancel = FALSE;irp->CancelRoutine = NULL;
…if (irp->Cancel) { IoCompleteIrp(irp);} else { IoSetCancelRoutine(irp, CancelRoutine); IoMarkIrpPending(irp);}…
…irp->Cancel = TRUE;fn = IoSetCancelRoutine(Irp, NULL);if (fn) { fn(irp);}…
void CancelRoutine(IRP *irp) { IoCompleteIrp(irp);}
Normal Cancellation
IO_REQUEST_PACKET *irp;
irp->Cancel = FALSE;irp->CancelRoutine = NULL;
…if (irp->Cancel) { IoCompleteIrp(irp);} else { IoSetCancelRoutine(irp, CancelRoutine); IoMarkIrpPending(irp);}…
…irp->Cancel = TRUE;fn = IoSetCancelRoutine(Irp, NULL);if (fn) { fn(irp);}…
void CancelRoutine(IRP *irp) { IoCompleteIrp(irp);}
Normal Cancellation
IO_REQUEST_PACKET *irp;
irp->Cancel = FALSE;irp->CancelRoutine = NULL;
…if (irp->Cancel) { IoCompleteIrp(irp);} else { IoSetCancelRoutine(irp, CancelRoutine); IoMarkIrpPending(irp);}…
…irp->Cancel = TRUE;fn = IoSetCancelRoutine(Irp, NULL);if (fn) { fn(irp);}…
void CancelRoutine(IRP *irp) { IoCompleteIrp(irp);}
Normal Cancellation
Fatal error!
IO_REQUEST_PACKET *irp;
irp->Cancel = FALSE;irp->CancelRoutine = NULL;
Concurrent programming is difficult
• Multiple loci of control resulting in non-local control flow
• Code difficult to understand and review
What about verification?
• Assertion-based modular reasoning becomes complicated due to non-local interactions– Floyd-Hoare morphs into Owicki-Gries
• Even with simple (finite) abstractions, the presence of concurrency makes the analysis computationally very expensive
Sequential Concurrent
Single Procedure
P-time PSPACE-complete
Multi Procedure
P-time Undecidable
What about testing?
• Number of executions = O( nnk ) • Exponential in both n and k
x = 1; … … … … … x = k;
x = 1; … … … … …x = k;
…
Thread 1 Thread nScheduling nondeterminism
• Uncontrollable
• Unobservable
• Exponential
Concurrency is important
• More than ever before
• Increasing importance of communicating systems– networked devices– cyber-physical systems
• Distributed programs running on the cloud– EC2, Azure, AppEngine, …
• Parallel programs running on multicores and GPUs– TBB, TPL, CUDA, AMP, …
Concurrency testing with CHESS
• Deterministic scheduling– make scheduling choices observable and
controllable
• Search prioritization– combating the combinatorial explosion of possible
schedules
Deterministic scheduling
Kernel: Threads, Scheduler, Synchronization Objects
While(not done){ TestScenario()}
TestScenario(){ …}
Program
CHESS
Win32 API
Tester Provides a Test Scenario
CHESS runs the scenario in a loop • Each run is a different interleaving• Each run is repeatable
Search prioritization (I)
• Given p ≥ 0, generate all schedules with up to p preemptions
• Pseudo-polynomial number of schedules– polynomial in preemption bound and schedule
points– exponential in number of threads
• Many bugs with fewer than 2 preemptions • Simple error traces for easier debugging
Search prioritization (II)
• Given p ≥ 0 and deterministic schedulers S0, …, Sp-1, schedule according to S0, …, Sp-1 in sequence moving from one to next nondeterministically– e.g., round-robin non-preemptive scheduling with p
different round-robin orders• Polynomial number of schedules• Testers can innovate by designing domain-
specific deterministic schedulers
CHESS is available
• Used internally by Microsoft product groups and externally by Microsoft customers
• Binary and source code available at:– http://chesstool.codeplex.com
Limitations of CHESS
• Exposing and gaining control of scheduling choices is difficult– most implementation effort and user frustration
due to this problem
• Testing components that interact extensively with the environment is difficult
• Input coverage is not addressed
Static program exploration with Poirot
• Symbolic instead of concrete execution
• C: Source code for software component• E: Model for environment and scheduler
• Explore behaviors of C+E– for all symbolic inputs– for all scheduling choices
Disk
AsyncRead(…) {
}
DiskReader(…) {
}
DiskReader(…) {
}
headtail
Request queue In-memory cache
cachecacheSize
Demo: Asynchronous File I/O
Poirot architecture
Trace Viewer
Concurrent .NET Program
Concurrent Boogie Program
Coverage Report
.NET Boogie
Concurrent C Program
C Boogie
Corral
Sequentialization Stratified Search
Error Trace
Concurrent Boogie Program
SequentialBoogie Program
Coverage Report
Searching with Corral
Refinement
Abstraction
Concurrent Boogie Program
Abstraction
• Set of global variables G
• Set of tracked variables T
• Drop writes to variables in G-T
• Replace reads to variables in G-T with nondeterministic values
Sequentialization Stratified Search
Error Trace
Concurrent Boogie Program
SequentialBoogie Program
Coverage Report
Searching with Corral
Refinement
Abstraction
Concurrent Boogie Program
Refinement
• Path p– feasible if only variables in T are tracked– infeasible if all variables in G are tracked
• Expand tracked set T to U such that p infeasible while tracking only variables in U
• Naïve algorithm: linear scan of G-T• New divide-and-conquer algorithm
– best case log(|G-T|)– worst case 2*|G-T|
Sequentialization Stratified Search
Error Trace
Concurrent Boogie Program
SequentialBoogie Program
Coverage Report
Searching with Corral
Refinement
Abstraction
Concurrent Boogie Program
Sequentialization (I)
• Given a concurrent program P, construct a sequential program Q such that Q P
• Drop each occurrence of async-call
• Convert each occurrence of async-call to call
Sequentialization (II)
• Given a concurrent program P, construct a family of programs Qi such that – Q0 Q1 Q2 … P
– iQi = P
• Even better if interesting behaviors of P manifest in Qi for low values of i
Context-bounding
• Under-approximation parameterized by K ≥ 0– executions in which each thread gets at most K
contexts to execute
• As K , we get all behaviors
• Can we create sequentializations for context-bounding?
Sequentializing context switches
Shared Memory
T1 T2
Local Memory Local Memory
Execution: T1 T2 T1 T2 T1
T1 T2 T1(s1, l1) (s2, l2) s2l2
Guess and verify
T1 T2T1(s1, l1) (s2, l2) (s3, l2)
Guess the effect of T2 Verify the guess
• Make copies of global variables• Source-to-source translation
– linear in program size and K• Generalizes to dynamically-created threads
Sequentialization Stratified Search
Error Trace
Concurrent Boogie Program
SequentialBoogie Program
Coverage Report
Searching with Corral
Refinement
Abstraction
Concurrent Boogie Program
Stratified search
main
…
Call tree given recursion bound r
VC(T)
assert no bugSummaries(L)T
L
assert no bug
VC(p)
Convert loops to recursive calls
Poirot status
• Medium-sized C programs– up to 20K low-level systems code– reports precise traces at scale
• Small .NET programs– bytecode to Boogie translator in progress
• Try: http://www.rise4fun.com/Poirot • Download available
Why bounded search?Data: Boolean, Integers, ArraysControl: Sequencing, Choice, Iteration, Call, Async-Call
SequencingChoice
NP-complete
SequencingChoiceIterationCallAsync-call
Undecidable
SequencingChoiceIterationCallAsync-call
+ bound
DecidablePSPACE-hard
Advances in SAT/SMT-solvers have made this problem tractableHAVOC verifier deployed for security analysis in Windows/IE
Rationale: It is better to fail at the simpler problem!
Poirot collaborators
Akash Lal, MSR Bangalore
Shuvendu Lahiri, MSR Redmond
Questions