One Thousand and One Refinement: From CDCL to a Verified...
Transcript of One Thousand and One Refinement: From CDCL to a Verified...
![Page 1: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/1.jpg)
One Thousand and One Refinement:From CDCL to a Verified SAT Solver
Mathias Fleury
2020/01/28
![Page 2: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/2.jpg)
When you start your proof
1/18
![Page 3: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/3.jpg)
After a few days…
1/18
![Page 4: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/4.jpg)
After a few days…Mistake!
1/18
![Page 5: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/5.jpg)
Then you write your paper
1/18
![Page 6: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/6.jpg)
Paper accepted = Proof correct
1/18
![Page 7: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/7.jpg)
Then you extend your paper
1/18
![Page 8: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/8.jpg)
Paper accepted = Proof correct
1/18
![Page 9: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/9.jpg)
What about ITPs?
When you start...
Before you finish
2/18
![Page 10: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/10.jpg)
What about ITPs?
When you start... Before you finish
2/18
![Page 11: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/11.jpg)
State of the art
Paper proofs vs proof assistants
2/18
![Page 12: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/12.jpg)
λ→
∀=Is
abelle
β
α
IsaFoL projectIsabelle Formalisation of Logic
3/18
![Page 13: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/13.jpg)
The IsaFoL project: motivation
Eat your own dog food
• case study for proof assistants and automatic provers
Build state-of-the-art libraries
• Automated Reasoning: The Art ofGeneric Problem Solving(ongoing textbook project by Christoph Weidenbach)
Focus onmeta-theorems
• reuse proofs
• be general
3/18
![Page 14: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/14.jpg)
The IsaFoL project: content
Excerpts of the IsaFoL project:
• Resolution, ordered resolution, and prover by Schlichtkrull et al.[ITP’16, IJCAR’18, CPP’19]
• Superposition by Peltier [AFP’16]
• UNSAT Checker by Lammich [CADE 27]
• CDCL and SAT solver [IJCAR’16, JAR’16, IJCAI’17, CPP’19,NFM’19]
3/18
![Page 15: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/15.jpg)
The IsaFoL project: content
Excerpts of the IsaFoL project:
• Resolution, ordered resolution, and prover by Schlichtkrull et al.[ITP’16, IJCAR’18, CPP’19]
• Superposition by Peltier [AFP’16]
• UNSAT Checker by Lammich [CADE 27]
• CDCL and SAT solver [IJCAR’16, JAR’16, IJCAI’17, CPP’19,NFM’19]
3/18
![Page 16: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/16.jpg)
Outline
CDCL
CDCL
Watched literals, heuristicsfancy data structure
Generated codeexecutable
36%
Heuristics
33%WL
21%
CDCL
10%
Code
Formalisation length (total:78 000 lines of code)
![Page 17: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/17.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠
†
♣
†
♥
assignement
= trail
clauses
4/18
![Page 18: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/18.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠
†
♣
†
♥
assignement
= trail
clauses
4/18
![Page 19: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/19.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠
†
♣
†
♥
assignement
= trail
clauses
4/18
![Page 20: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/20.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠
†
♣
†
♥
assignement = trail clauses
4/18
![Page 21: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/21.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠†
♣
†
♥
assignement = trail clauses
4/18
![Page 22: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/22.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠†
♣
†
♥
assignement = trail clauses
4/18
![Page 23: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/23.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠
†
♣
†
♥
assignement = trail clauses
4/18
![Page 24: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/24.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠
†
♣†
♥
assignement = trail clauses
4/18
![Page 25: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/25.jpg)
CDCL explanation
♣ ♠
♥
♣ ♠
♣ ♠ ♣
♥
♠
†
♣†
♥
assignement = trail clauses
4/18
![Page 26: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/26.jpg)
Refinement by specialisation
Core of CDCL is DPLL+BJ
DPLL+BJ = Propagate + Decide + Backjump
back to some decision
DPLL = Propagate + Decide + Backtrack
back to latest decision
⊆
How tomaximize reuse?
Backtrack = Parametrised Backjump (Backtrack_cond)
5/18
![Page 27: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/27.jpg)
Refinement by specialisation
Core of CDCL is DPLL+BJ
DPLL+BJ = Propagate + Decide + Backjump
back to some decision
DPLL = Propagate + Decide + Backtrack
back to latest decision
⊆
How tomaximize reuse?
Backtrack = Parametrised Backjump (Backtrack_cond)
5/18
![Page 28: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/28.jpg)
Backjump on paper vs. in Isabelle
Backjump on paper
if C ∈ N andM ⊨ ¬C and there is a C′ such that...then (M,N) ⇒CDCL (M
′L,N).
Definition (Parametrised Backjump in Isabelle)
if C ∈ N andM ⊨ ¬C and there is a C′ such that...and BJ_cond C′
then (M,N) ⇒CDCL (M′L,N).
6/18
![Page 29: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/29.jpg)
Development hierarchy
DPLL+BJ
DPLL
CDCL
CDCL+learn_BJ CDCL+restart
CDCL+learn_BJ+restart
CDCL+learn_BJ+restart+T
specialises
extends
refines extends
extends refines
extends
sublocale DPLL < DPLL+BJ whereDPLL+BJ_Cond = DPLL_Cond
CDCL = DPLL+BJ + Learn + Forget
Strategy used in most implementations:learn only backjump clause
7/18
![Page 30: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/30.jpg)
Development hierarchy
DPLL+BJ
DPLL CDCL
CDCL+learn_BJ CDCL+restart
CDCL+learn_BJ+restart
CDCL+learn_BJ+restart+T
specialisesextends
refines extends
extends refines
extends
sublocale DPLL < DPLL+BJ whereDPLL+BJ_Cond = DPLL_Cond
CDCL = DPLL+BJ + Learn + Forget
Strategy used in most implementations:learn only backjump clause
7/18
![Page 31: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/31.jpg)
Development hierarchy
DPLL+BJ
DPLL CDCL
CDCL+learn_BJ
CDCL+restart
CDCL+learn_BJ+restart
CDCL+learn_BJ+restart+T
specialisesextends
refines
extends
extends refines
extends
sublocale DPLL < DPLL+BJ whereDPLL+BJ_Cond = DPLL_Cond
CDCL = DPLL+BJ + Learn + Forget
Strategy used in most implementations:learn only backjump clause
7/18
![Page 32: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/32.jpg)
Development hierarchy
DPLL+BJ
DPLL CDCL
CDCL+learn_BJ CDCL+restart
CDCL+learn_BJ+restart
CDCL+learn_BJ+restart+T
specialisesextends
refines extends
extends refines
extends
sublocale DPLL < DPLL+BJ whereDPLL+BJ_Cond = DPLL_Cond
CDCL = DPLL+BJ + Learn + Forget
Strategy used in most implementations:learn only backjump clause
7/18
![Page 33: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/33.jpg)
Weidenbach’s CDCL
Definition (Parametrised Backjump (BJ_cond))
if C ∈ N andM ⊨ ¬C and there is a C′ such that...and BJ_cond C′
then (M,N) ⇒CDCL (L†M′,N).
How to get a suitable C′?
8/18
![Page 34: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/34.jpg)
Refinement by inclusion
CDCL_learn_BJ
Decide, propagate Backjump+ Learn
terminating
terminating
CDCL_W
Decide, propagateConflict
Skip and resolveJump+Learn
M N
M N U D
M N U
9/18
![Page 35: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/35.jpg)
Refinement by inclusion
CDCL_learn_BJ
Decide, propagate Backjump+ Learn
terminating
terminating
CDCL_W
Decide, propagateConflict
Skip and resolveJump+Learn
(M,N)
(M,N,U,D)
(M,N + U)
9/18
![Page 36: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/36.jpg)
Refinement by inclusion
CDCL_learn_BJ
Decide, propagate Backjump+ Learn
terminating
terminatingCDCL_W
Decide, propagateConflict
Skip and resolveJump+Learn
M N
M N U D
M N U
9/18
![Page 37: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/37.jpg)
Outline
Refining Data Structures
CDCL
Watched literals, heuristicsfancy data structure
Generated codeexecutable
36%
Heuristics
33%WL
21%
CDCL
10%
Code
Formalisation length (total:78 000 lines of code)
![Page 38: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/38.jpg)
Watched literals explanation
Watched literals = sophisticated data structure to identifypropagations and conflicts.
♣♦ ♠
♦♣
♣ ♠
♦
♣
♣♦
♥
10/18
![Page 39: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/39.jpg)
Watched literals explanation
Watched literals = sophisticated data structure to identifypropagations and conflicts.
♣♦ ♠
♦♣
♣ ♠
♦
♣
♣♦
♥
10/18
![Page 40: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/40.jpg)
Watched literals explanation
Watched literals = sophisticated data structure to identifypropagations and conflicts.
♣♦ ♠
♦♣
♣ ♠
♦
♣
♣♦
♥
10/18
![Page 41: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/41.jpg)
Watched literals explanation
Watched literals = sophisticated data structure to identifypropagations and conflicts.
♣♦ ♠
♦♣
♣ ♠
♦
♣
♣♦
♥
10/18
![Page 42: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/42.jpg)
Watched literals
First formalisation attempt failed.
Development done in two steps:
1. watched literals...
2. ... extended with blocking literals
My Approach non-deterministic transition system
11/18
![Page 43: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/43.jpg)
Watched literals
First formalisation attempt failed.
Development done in two steps:
1. watched literals...
2. ... extended with blocking literals
My Approach non-deterministic transition system
11/18
![Page 44: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/44.jpg)
Watched literals
First formalisation attempt failed.
Development done in two steps:
1. watched literals...
2. ... extended with blocking literals
My Approach non-deterministic transition system
11/18
![Page 45: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/45.jpg)
Refinement in the non-determinismmonad
Then we enter the non-determinismmonad:
• closer to programs
• preserves non-determinism
Abstract level:
OBTAIN should_restart such that
should_restart conflict threshold
Concrete level:
should_restart RETURN conflict thresholdheuristic
12/18
![Page 46: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/46.jpg)
Refinement in the non-determinismmonad
Then we enter the non-determinismmonad:
• closer to programs
• preserves non-determinism
Abstract level:
OBTAIN should_restart such that
should_restart ⟹ #conflict > threshold
Concrete level:
should_restart RETURN conflict thresholdheuristic
12/18
![Page 47: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/47.jpg)
Refinement in the non-determinismmonad
Then we enter the non-determinismmonad:
• closer to programs
• preserves non-determinism
Abstract level:
OBTAIN should_restart such that
should_restart ⟹ #conflict > threshold
Concrete level:
should_restart ← RETURN(#conflict > threshold∧heuristic)
12/18
![Page 48: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/48.jpg)
Refinement to keep abstractions
CDCLWeidenbach
♣ ♠
♦
Watched clauseswith multisets
♣ ♠
♦
Watched clausesclauses as lists
[♣,♠,♦]
Watch listsefficient indexing
Isasatdeterministic with heuristics
0 2 5
13/18
![Page 49: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/49.jpg)
Refinement to keep abstractions
CDCLWeidenbach
♣ ♠
♦
Watched clauseswith multisets
♣ ♠
♦
Watched clausesclauses as lists
[♣,♠,♦]
Watch listsefficient indexing
Isasatdeterministic with heuristics
0 2 5
13/18
![Page 50: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/50.jpg)
Outline
Generating Code
CDCL
Watched literals, heuristicsfancy data structure
Generated codeexecutable
36%
Heuristics
33%WL
21%
CDCL
10%
Code
Formalisation length (total:78 000 lines of code)
![Page 51: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/51.jpg)
What is the imperative code?
IsaSATdeterministic
Imperative IsaSATin Imperative HOL
IsaSAT/Standard ML (SML)or Scala, OCaml, Haskell
automatic synthesis
automatic mapping
14/18
![Page 52: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/52.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i
After printing in SML, via code equations and printing:Array.sub(xs, i)
if i < Array.size xsthen xs[i]else raise OutOfBound
15/18
![Page 53: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/53.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i
After printing in SML, via code equations and printing:Array.sub(xs, i)
if i < Array.size xsthen xs[i]else raise OutOfBound
15/18
![Page 54: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/54.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i
After printing in SML, via code equations and printing:Array.sub(xs, i)
if i < Array.size xsthen xs[i]else raise OutOfBound
In IsaSAT removedby a compiler flag...
15/18
![Page 55: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/55.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i
After printing in SML, via code equations and printing:Array.sub(xs, i)
if i < Array.size xsthen xs[i]else raise OutOfBound
A native array
In IsaSAT removedby a compiler flag...
15/18
![Page 56: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/56.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i
After printing in SML, via code equations and printing:if i < Array.size xsthen xs[i]else raise OutOfBound
In IsaSAT removedby a compiler flag...
15/18
![Page 57: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/57.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
Information is lostduring translation
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i
After printing in SML, via code equations and printing:if i < Array.size xsthen xs[i]else raise OutOfBound In IsaSAT removed
by a compiler flag...15/18
![Page 58: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/58.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
In the nice Isabelleworld GMP integer
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i
After printing in SML, via code equations and printing:if i < Array.size xsthen xs[i]else raise OutOfBound
In IsaSAT removedby a compiler flag...
15/18
![Page 59: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/59.jpg)
Code synthesis and generation
Abstract code:ASSERT(i < length xs);RETURN(xs!i);
In IsaSAT, uint64 in-teger until it doesnot fit
After synthesis by Sepref in Imperative HOL:Array.nth xs i Array.nth_uint64 xs i
After printing in SML, via code equations and printing:if i < Array.size xsthen xs[i]else raise OutOfBound
In IsaSAT removedby a compiler flag...
Array.nth_uint64(xs, i)
15/18
![Page 60: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/60.jpg)
Correctness theorem
TheoremIf the input is well formed and UNSAT (resp. SAT), then IsaSATterminates and it returns UNSAT (resp. SAT with amodel).1
And the only other efficient verified solver
Theorem (Correctness versat)If the input is well formed and the solver returns UNSAT, then theproblem is UNSAT.
1if the Standard ML compiler is able to allocate large enough arrays
16/18
![Page 61: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/61.jpg)
Correctness theorem
TheoremIf the input is well formed and UNSAT (resp. SAT), then IsaSATterminates and it returns UNSAT (resp. SAT with amodel).1
And the only other efficient verified solver
Theorem (Correctness versat)If the input is well formed and the solver returns UNSAT, then theproblem is UNSAT.
1if the Standard ML compiler is able to allocate large enough arrays
16/18
![Page 62: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/62.jpg)
Performance
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●
●●
●●
●●●
●●
●
●
●●●
●
●●
●
●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●
●
●
●
●
versatIsaSAT (SML)IsaSAT (LLVM)CaDiCaLminisat
Comparison of various SAT solvers on preprocessed instances 17/18
![Page 63: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/63.jpg)
Performance
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●
●●
●●
●●●
●●
●
●
●●●
●
●●
●
●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●
●
●
IsaSAT (LLVM)CaDiCaLminisat
Comparison of various SAT solvers on preprocessed instances 17/18
![Page 64: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/64.jpg)
Performance
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●
●
●
IsaSAT (LLVM)CaDiCaLminisat
Comparison of various SAT solvers on preprocessed instances 17/18
![Page 65: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/65.jpg)
Conclusion
![Page 66: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/66.jpg)
Conclusion
CDCLextensible
Optimizing CDCLCDCL+brand-and-bound
Watched Literalsfancy data structure
Generated Codeexecutable
18/18
![Page 67: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/67.jpg)
Conclusion
CDCLextensible
Optimizing CDCLCDCL+brand-and-bound
Watched Literalsfancy data structure
Generated Codeexecutable
18/18
![Page 68: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/68.jpg)
O Captain! My Captain!Now comes the appendix, go back to the previous
slide
18/18
![Page 69: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/69.jpg)
Appendix Outline
What is hard?
Refinement
Correctness and Trust
Features
Missing Features
CDCL
Complexity
Importing Correctness in Isabelle
IsaSAT/LLVM vs IsaSAT/MLton
Performance
OCDCL
Related Work
![Page 70: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/70.jpg)
What is hard?
![Page 71: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/71.jpg)
Why is it so hard?
![Page 72: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/72.jpg)
Size
Mostly about defi-nitions
Formalisation part Length (kloc)
CDCL Libraries 3 EntailmentCDCL 17Refinement Libraries 6 Setup for machine
words, arrays of ar-rays
Refinement except last layer 26Heuristics 35 code synthesis, lots
of code
![Page 73: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/73.jpg)
Size
Mostly about defi-nitions
Formalisation part Length (kloc)
CDCL Libraries 3 EntailmentCDCL 17Refinement Libraries 6 Setup for machine
words, arrays of ar-rays
Refinement except last layer 26Heuristics 35 code synthesis, lots
of code
![Page 74: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/74.jpg)
Size
Mostly about defi-nitions
Aliasing and owner-ship
Formalisation part Length (kloc)
CDCL Libraries 3 EntailmentCDCL 17Refinement Libraries 6 Setup for machine
words, arrays of ar-rays
Refinement except last layer 26Heuristics 35 code synthesis, lots
of code
![Page 75: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/75.jpg)
Size
Mostly about defi-nitions
Aliasing and owner-ship
Formalisation part Length (kloc)
CDCL Libraries 3 EntailmentCDCL 17Refinement Libraries 6 Setup for machine
words, arrays of ar-rays
Refinement except last layer 26Heuristics 35 code synthesis, lots
of code
![Page 76: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/76.jpg)
Size
Mostly about defi-nitions
Aliasing and owner-ship
Single threadedFormalisation part Length (kloc)
CDCL Libraries 3 EntailmentCDCL 17Refinement Libraries 6 Setup for machine
words, arrays of ar-rays
Refinement except last layer 26Heuristics 35 code synthesis, lots
of code
![Page 77: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/77.jpg)
Refinement
![Page 78: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/78.jpg)
Refinement in the non-determinismmonad: Data structure
Abstract level:
OBTAIN L s.t. L ∈ C
Concrete level:
blit ← RETURN(watcher.blit)
![Page 79: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/79.jpg)
Correctness and Trust
![Page 80: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/80.jpg)
Correctness
And IsaSAT/LLVM:
Theorem (Correctness IsaSAT/LLVM)If the input is a valid input and the solver returns SAT (UNSAT), thenthe problem is SAT (UNSAT).
![Page 81: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/81.jpg)
Isabelle protects of:
• programming errors (out-of-bound)
• correctness errors (SAT instead of UNSAT)
But not of:
• performance bugs (restarts)
![Page 82: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/82.jpg)
What do you trust?
IsaSAT/SML IsaSAT/LLVM CaDiCaL
Parser Parser The parserCDCL
Code equations Isabelle’s LLVM Se-mantics
Implementation
Compiler LLVM Compiler∼2 faster than SML,∼10 times lessmem-ory
There is no bug that happens after two years of calculation becauseyou wrote uint64_max - 4 instead of uint64_max - 5
![Page 83: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/83.jpg)
What do you trust?
IsaSAT/SML IsaSAT/LLVM CaDiCaL
Parser Parser The parserCDCL
Code equations Isabelle’s LLVM Se-mantics
Implementation
Compiler LLVM Compiler∼2 faster than SML,∼10 times lessmem-ory
There is no bug that happens after two years of calculation becauseyou wrote uint64_max - 4 instead of uint64_max - 5
![Page 84: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/84.jpg)
Features
![Page 85: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/85.jpg)
Techniques in IsaSAT
VMTF decision heuristic CriticalConflicts as hash-table and array CriticalRecursive conflict minimization Critical
Arena-basedmemory I never saw a differenceBlocking literals + position saving Helps a lotEMA-14 restarts + trail reuse Helps, but I still don’t understand
what CaDiCaL doesSpecial handling of binary clauses I never saw a difference
![Page 86: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/86.jpg)
Missing Features
![Page 87: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/87.jpg)
Missing Features
Two trivial but key features
• deletion of true clauses
• removal of false literals
Solution: “pragmatic CDCL” with resolution rules to simplify clausesset
![Page 88: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/88.jpg)
CDCL
![Page 89: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/89.jpg)
Is Weidenbach’s CDCL the right CDCL?
Easy to add:
Definition (Conflict Minimisation)
Learn a clause D′ ∨ L′ ⊆ D ∨ L if N ⊨ D′ ∨ L′.
Impossible to add (it breaks invariants):
Definition (Inprocessing)An irredundant clause is subsumed by a learned clause: make thelatter irredundant.
![Page 90: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/90.jpg)
but!
If we go with(M,N,Nsubsumed,U,Usubsumed,D)
and do not consider subsumed clauses, CDCL can see
(M,N + Nsubsumed,U + Usubsumed,D)
and everything will work as expected.
![Page 91: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/91.jpg)
Complexity
![Page 92: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/92.jpg)
Complexity
As for SAT implementations,
Never-ending task there is always onemore heuristic or one moretechnique to implement...
No tooling ... makes it even harder
Testing a heuristic is hard
![Page 93: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/93.jpg)
Complexity
On the proof side
Proving Correctness time consuming (overflow problems), Isabelleis slow
Side conditions of CDCL
Property (CDCL Invariant)The set of all literals you consider is exactly the set of literals in the setof clauses.
![Page 94: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/94.jpg)
Evaluator PerformanceMLton 2.5 s includes parsingPolyML 43 svalue ? requires 64-bit PolyML
nbe, simp ⊥ do not know about Imperative HOL
![Page 95: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/95.jpg)
What makes refinement hard?
Refinement is easy when:
• you can ignore the result of operations
• i.e., reduce interdependency between components of the state
M <- RETURN (Decided L . M)What is the impact on the other components?
![Page 96: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/96.jpg)
What makes refinement hard?
trail clausesconflictclause
decisionheuristic
everyliteralis in
everyliteralis in
![Page 97: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/97.jpg)
What makes refinement hard?
trail clausesconflictclause
decisionheuristic
everyliteralis in
everyliteralis in
![Page 98: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/98.jpg)
Importing Correctness in Isabelle
![Page 99: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/99.jpg)
Idea
Abstract code:
ASSERT(i < length xs);RETURN (xs ! i);
After synthesis, done automatically by Sepref:
return xs[i]
Can we run it in Isabelle?
• result cannot be extracted from the return (imperativemonad)...
• ... but we can generate a purely functional version...
• ... which is what I optimised for
![Page 100: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/100.jpg)
Evaluator PerformanceMLton 2.5 s includes parsingPolyML 43 svalue ? requires 64-bit PolyML
nbe, simp ⊥ donot knowabout ImperativeHOL,so you cannot allocate arrays
![Page 101: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/101.jpg)
IsaSAT/LLVM vs IsaSAT/MLton
![Page 102: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/102.jpg)
LLVM is better and has an easier job
• LLVM has more man-power: MLton’s LLVM backend producesslightly better code
• LLVM’s IR is the target for tools vs target for humans (Isabelle’scode generator produces terrible and unreadable code)
• LLVM’s input is the code you would expect
![Page 103: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/103.jpg)
LLVM hasmore freedom to do a good job
• The code is not functional at all and contains barely anydatatype
• ML enforces sharing, which is good until is not1. λ(#props, stats). (#props + 1, stats) reallocates2. clause_ref * (bool * literal)2 needs more
memory than struct {clause_ref; struct {bool;literal};} (cache problems!)
• Array access and conversions are checked3
2Isabelle is not able to generate clause_ref * bool * literal and usinga tuple made things worse3although I deactivate these checks
![Page 104: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/104.jpg)
Memory is not cheap
• IsaSAT/ML uses 10 times more memory
• IsaSAT/ML uses the GC... but I have no idea why: IsaSAT usesbase types (or with in-place operations) and arrays resizing(freeing the old one is enough)
![Page 105: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/105.jpg)
Performance
![Page 106: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/106.jpg)
Performance
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●
●●
●●
●●●
●●
●
●
●●●
●
●●
●
●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0 500 1000 1500 2000
0500
1000
1500
2000
#solved
tim
e
●
●
●
●
●
●
●
●
●
●
versatIsaSAT (SML)IsaSAT (LLVM)versatCaDiCaLIsaSAT (LLVM, latest)microsatcrypominisatglucoseminisat
Comparison of various SAT solvers on preprocessed instances
![Page 107: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/107.jpg)
OCDCL
![Page 108: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/108.jpg)
Lemma (verified in Isabelle)
ODPLL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Conjecture
OCDCL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Problem: backjump is nearly a restart.
![Page 109: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/109.jpg)
Lemma (verified in Isabelle)
ODPLL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Conjecture
OCDCL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Problem: backjump is nearly a restart.
![Page 110: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/110.jpg)
Lemma (verified in Isabelle)
ODPLL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Conjecture
OCDCL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Problem: backjump is nearly a restart.
![Page 111: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/111.jpg)
Lemma (verified in Isabelle)
ODPLL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Conjecture
OCDCL+stgy performs at most3n Backtrack steps.
Proof.
• trails are not repeated
• trails have a certain form
• and they are such 3n suchtrail
Problem: backjump is nearly a restart.
![Page 112: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/112.jpg)
Related Work
![Page 113: One Thousand and One Refinement: From CDCL to a Verified ...fmv.jku.at/fleury/talk/Fleury-Defense-talk.pdf · TheIsaFoLproject:content ExcerptsoftheIsaFoLproject: • Resolution,orderedresolution,andproverbySchlichtkrulletal.](https://reader034.fdocuments.us/reader034/viewer/2022042411/5f2a43c5a1efb41c1b61751a/html5/thumbnails/113.jpg)
Related Work
Marić Les-cuyer
Schankaret al
Oe et al
2008
Isabelle
2011 Coq 2011 PVS 2012Guru
BackjumpingLearning - *SoundnessCompeleteness -Implementa-tion
-
Termination -Restart+Forget - - - -WL ∼ - - ∼