Post on 29-May-2020
ESOP
March 19, 2013
Mihai Budiu, Microsoft Research, Silicon Valley
Joel Galenson, UC Berkeley
Gordon Plotkin, University of Edinburgh
• Motivating example
–Declarative parallel programming with LINQ and DryadLINQ
• Divide-and-conquer compilation
–Compilers and Partial Compilers
• Building real compilers – LINQ, DryadLINQ, and matrix computations
2
LINQ Summary
Input
3
.Net objects of type T
LINQ Summary
Where (filter)
Input
4
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
Input
5
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
Input
6
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
OrderBy (sort)
Input
7
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
OrderBy (sort)
Aggregate (fold)
Input
8
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
OrderBy (sort)
Aggregate (fold)
Join
Input
9
.Net objects of type T
Distributed Collections
10
Distributed Collections
11
Partition
Distributed Collection
Select (Map)
f( )=
12
Select (Map)
f( )=
Select(x => f(x))
13
Select (Map)
f( )=
Select(x => f(x))
+
14
Select (Map)
f( )=
Select(x => f(x))
+
+
15
Aggregate (Fold)
f( , )=
associative
16
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
associative
17
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
+
associative
18
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
+
associative
+
19
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
+
associative
+
20
GroupBy
GroupBy(x => K(x))
21
GroupBy
GroupBy(x => K(x))
o
22
GroupBy
GroupBy(x => K(x))
o
o
PartitionBy(x => K(x))
23
GroupBy
GroupBy(x => K(x))
o
o
o
PartitionBy(x => K(x))
GroupBy(x => K(x))
Equal under permutation
24
• Motivating example
–Declarative parallel programming with LINQ and DryadLINQ
• Divide-and-conquer compilation
–Compilers and Partial Compilers
• Building real compilers – LINQ, DryadLINQ, and matrix computations
25
Compilers
Compiler
S T
26
𝐶 ∶ source → target
Partial compilers
Child
Compiler
target source
source’ target’
Partial compiler
27
Partial compilers
Child
Compiler
target source
source’ target’
Partial compiler
28
𝑃𝐶 ∶ source → source′ × target′ → target
Partial compilers
Child
Compiler
target source
source’ target’
Partial compiler R
re
du
ce G
gen
erat
e
29
𝑃𝐶 ∶ source → source′ × target′ → target
Compiler Composition target source
source’ target’
PC
C
30
Compiler Composition target source
source’ target’
PC
C
31
PC’
Partial Compiler Composition
PC
32
target source
source’ target’
source’’ target’’
Composition Laws
33
Composition Laws
Id
34
Compilers as First-Class Objects
PC PC’ ⊗
Tensor: 𝑃𝐶 ⊗ 𝑃𝐶′ 35
Multiple Children
SEQ
C1 C2
S T
36
Multiple Children
SEQ
C1 C2
Example: 𝑆 = 𝑓 ∘ 𝑔 𝑇 = 𝐶1 𝑓 ∘ 𝐶2(𝑔)
S T
37
Star
38
C
*
source* target*
Star
39
C
*
source* target*
Star
40
C
*
source* target* … …
Conditional
if gpu
GPU CPU
gpu : source -> bool
41
Case
42
case
Select GroupBy Aggregate
• Motivating example
–Declarative parallel programming with LINQ and DryadLINQ
• Divide-and-conquer compilation
–Compilers and Partial Compilers
• Building real compilers – LINQ, DryadLINQ, and matrix computations
43
A LINQ Compiler
44
Select GroupBy Aggregate
A LINQ Compiler
45
case
Select GroupBy Aggregate
A LINQ Compiler
46
case
Select GroupBy Aggregate
*
A LINQ Compiler
47
case
Select GroupBy Aggregate
*
SEQ
A Multi-Core LINQ Compiler
48
case
Select GroupBy Aggregate
MC
*
SEQ
A Multi-Core LINQ Compiler
49
case
Select GroupBy Aggregate
MC
*
SEQ
MC
A Multi-Core LINQ Compiler
50
case
Select GroupBy Aggregate
MC
*
SEQ
MC
S
T
A Multi-Core LINQ Compiler
51
case
Select GroupBy Aggregate
MC
*
SEQ
MC
S
o
o
T
T T
A Distributed, Multi-Core
LINQ Compiler
52
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
A Distributed, Multi-Core
LINQ Compiler
53
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
Cluster
A Distributed, Multi-Core
LINQ Compiler
54
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
Cluster
S
T
A Distributed, Multi-Core
LINQ Compiler
55
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
Cluster
S
o
o
T
T T
A Distributed Matrix Compiler
56
*
A Distributed Matrix Compiler
57
Tile Tile *
Cluster
A Distributed Matrix Compiler
58
Matrix
*
SEQ
Tile
Tile Tile *
Cluster
A Distributed Matrix Compiler
59
Matrix
*
SEQ
case
+ * …
Tile Tile *
M1.Times(M2).Plus(M3)
60
M1
M2
M3
*
+
Join. GroupBy.
Select( Aggregate) Concat
GroupBy. Select(
Aggregate)
+
PCTILE
PCTILE
PCTILE
CLINQ
CLINQ
*
PCMATRIX
+
PCMATRIX
PCSEQ
M1.Times(M2).Plus(M3)
61
M1
M2
M3
*
+
Join. GroupBy.
Select( Aggregate) Concat
GroupBy. Select(
Aggregate)
+
PCTILE
PCTILE
PCTILE
CLINQ
CLINQ
*
PCMATRIX
+
PCMATRIX
PCSEQ
M1.Times(M2).Plus(M3)
62
M1
M2
M3
*
+
Join. GroupBy.
Select( Aggregate) Concat
GroupBy. Select(
Aggregate)
+
PCTILE
PCTILE
PCTILE
CLINQ
CLINQ
PCMATRIX
PCMATRIX
PCSEQ
M1.Times(M2).Plus(M3)
63
Partial Compiler Correctness
64
if gpu
GPU CPU
Partial Compiler Correctness
65
𝑔𝑝𝑢 𝑆 𝐶𝐺𝑃𝑈 {¬𝑔𝑝𝑢 𝑆 }𝐶𝐶𝑃𝑈
𝑇 𝐼𝐹 𝑔𝑝𝑢 𝑇𝐻𝐸𝑁 𝐶𝐺𝑃𝑈 𝑒𝑙𝑠𝑒 𝐶𝐶𝑃𝑈 if gpu
GPU CPU
Partial Compiler Correctness
66
𝑔𝑝𝑢 𝑆 𝐶𝐺𝑃𝑈 {¬𝑔𝑝𝑢 𝑆 }𝐶𝐶𝑃𝑈
𝑇 𝐼𝐹 𝑔𝑝𝑢 𝑇𝐻𝐸𝑁 𝐶𝐺𝑃𝑈 𝑒𝑙𝑠𝑒 𝐶𝐶𝑃𝑈
Totally correct compiler from partially correct parts!
if gpu
GPU CPU
Related Work
• Dialectica category
– Inspired partial compilers and their operators.
• Milner’s tactics
– Partial compilers are a typed form of tactics.
• Multistage compilers
– Fit as a composition of unary partial compilers.
• Federated databases, cooperating analyses
– Interesting applications.
67
Thank you
68
Backup slides
69
Homomorphisms
70
ℎ ∶ 𝐴 → 𝐵 ℎ 𝑎1 +𝐴 𝑎2 = ℎ 𝑎1 +𝐵 ℎ(𝑎2)
collections
concatenation
concatenation
Nested Parallelism
71
Machine
Nested Parallelism
72
Machine Core
Correctness Definition
73
⊨ ⊆ 𝑡𝑎𝑟𝑔𝑒𝑡 × 𝑠𝑜𝑢𝑟𝑐𝑒 𝑇 ⊨ 𝑆
T implements the meaning of S
𝐶 ∶ 𝑠𝑜𝑢𝑟𝑐𝑒 → 𝑡𝑎𝑟𝑔𝑒𝑡 𝐶 𝑖𝑠 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝑤. 𝑟𝑒𝑠𝑝. 𝑡𝑜 ⊨ iff
∀𝑆 ∈ 𝑠𝑜𝑢𝑟𝑐𝑒. 𝐶 𝑆 ⊨ 𝑆
C
T S
Partial Compiler Correctness
74
⊨ ⊆ 𝑠𝑜𝑢𝑟𝑐𝑒 × 𝑡𝑎𝑟𝑔𝑒𝑡 ⊨′ ⊆ 𝑠𝑜𝑢𝑟𝑐𝑒′ × 𝑡𝑎𝑟𝑔𝑒𝑡′
PC is correct iff ∀𝑆, 𝑇′. 𝑇′ ⊨′ 𝑆′ ⇒ 𝑇 ⊨ 𝑆
T S
T’ S’
Correctness Theorems
75
PC, C correct ⇒ correct
𝑃𝐶 ⊗ 𝑃𝐶′ correct
𝐶∗ correct
etc.
Partial Correctness
76
Correct only for some programs
Correct only for some programs
Generated sub-programs satisfy this predicate
Partial Correctness
77
Correct only for some programs
Correct only for some programs
Generated sub-programs satisfy this predicate
{Precondition} {Postcondition}
Functor
78
functor
CPU
Trans Id
Id : target -> target’ Trans : source -> source’
Iteration
79
iteration
Opt
A Distributed Matrix Compiler
80
Matrix -> sets of tiles
Matrix operations Distributed LINQ
Tile operations LINQ
Staged Compilers
81