Art of Multiprocessor Programming
Transcript of Art of Multiprocessor Programming
![Page 1: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/1.jpg)
Data Parallelism
Companion slides for
The Art of Multiprocessor Programming
by Maurice Herlihy & Nir Shavit
![Page 2: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/2.jpg)
Ever Wonder …
Art of Multiprocessor Programming 2
When did the term multicore become popular?
A multi-core processor is a single computing
component with two or more independent actual
central processing units, which are the units that
read and execute program instructions.)wikipedia
![Page 3: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/3.jpg)
Let’s Ask Google Ngram!
Art of Multiprocessor Programming 3
usage of multicore in books by publication year
![Page 4: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/4.jpg)
Let’s Ask Google Ngram!
Art of Multiprocessor Programming 4
This part since 2000 is obvious …
![Page 5: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/5.jpg)
Let’s Ask Google Ngram!
Art of Multiprocessor Programming 5
???
![Page 6: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/6.jpg)
Let’s Ask Google Ngram!
Art of Multiprocessor Programming 6
multicore cable
multicore fiber
…
but we digress …
![Page 7: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/7.jpg)
WordCount
Art of Multiprocessor Programming 7
alpha → 8
easy to do sequentially …what about in parallel?
bravo → 3
charlie → 9
…
zulu → 1
![Page 8: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/8.jpg)
MapReduce
Art of Multiprocessor Programming 8
…
chapter 1 chapter 2 chapter k
split text among mapping threads
![Page 9: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/9.jpg)
Map Phase
Art of Multiprocessor Programming 9
…
chapter 1 chapter 2 chapter k
must count
words!must count
words!
must count
words!
a mapping thread per chapter
![Page 10: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/10.jpg)
Map Phase
Art of Multiprocessor Programming 10
chapter 1
alpha → 9
juliet → 2,
alpha → 1
tango → 4
…each mapper thread produces a stream …of key-value pairs …
key: word
value: local count
![Page 11: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/11.jpg)
Art of Multiprocessor Programming 11
Mapper Class
abstract class Mapper<IN, K, V>
extends RecursiveTask<Map<K, V>> {
IN input;
public void setInput(IN anInput) {
input = anInput;
}
}
![Page 12: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/12.jpg)
Art of Multiprocessor Programming 12
Mapper Class
abstract class Mapper<IN, K, V>
extends RecursiveTask<Map<K, V>> {
IN input;
public void setInput(IN anInput) {
input = anInput;
}
}
input: document fragment
![Page 13: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/13.jpg)
Art of Multiprocessor Programming 13
Mapper Class
abstract class Mapper<IN, K, V>
extends RecursiveTask<Map<K, V>> {
IN input;
public void setInput(IN anInput) {
input = anInput;
}
}key: individual word
![Page 14: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/14.jpg)
Art of Multiprocessor Programming 14
Mapper Class
abstract class Mapper<IN, K, V>
extends RecursiveTask<Map<K, V>> {
IN input;
public void setInput(IN anInput) {
input = anInput;
}
}
value: local count
![Page 15: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/15.jpg)
Art of Multiprocessor Programming 15
Mapper Class
abstract class Mapper<IN, K, V>
extends RecursiveTask<Map<K, V>> {
IN input;
public void setInput(IN anInput) {
input = anInput;
}
}
a task that runs in parallel with other tasks
![Page 16: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/16.jpg)
Art of Multiprocessor Programming 16
Mapper Class
abstract class Mapper<IN, K, V>
extends RecursiveTask<Map<K, V>> {
IN input;
public void setInput(IN anInput) {
input = anInput;
}
}
produces a map: word → count
![Page 17: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/17.jpg)
Art of Multiprocessor Programming 17
Mapper Class
abstract class Mapper<IN, K, V>
extends RecursiveTask<Map<K, V>> {
IN input;
public void setInput(IN anInput) {
input = anInput;
}
}
initialize input: which document fragment?
![Page 18: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/18.jpg)
Art of Multiprocessor Programming 18
WordCount Mapperclass WordCountMapper extends
mapreduce.Mapper<
List<String>, String, Long
> {
…
}
![Page 19: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/19.jpg)
Art of Multiprocessor Programming 19
WordCount Mapperclass WordCountMapper extends
mapreduce.Mapper<
List<String>, String, Long
> {
…
}document fragment is
list of words
![Page 20: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/20.jpg)
Art of Multiprocessor Programming 20
WordCount Mapperclass WordCountMapper extends
mapreduce.Mapper<
List<String>, String, Long
> {
…
}document fragment is
list of wordsmap each word …
![Page 21: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/21.jpg)
Art of Multiprocessor Programming 21
WordCount Mapperclass WordCountMapper extends
mapreduce.Mapper<
List<String>, String, Long
> {
…
}document fragment is
list of wordsmap each word …
to its count in
the fragment
![Page 22: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/22.jpg)
Art of Multiprocessor Programming 22
WordCount MapperMap<String,Long> compute() {
Map<String,Long> map = new HashMap<>();
for (String word : input) {
map.merge(word,
1L,
(x, y) -> x + y);
}
return map;
}
}
![Page 23: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/23.jpg)
Art of Multiprocessor Programming 23
WordCount MapperMap<String,Long> compute() {
Map<String,Long> map = new HashMap<>();
for (String word : input) {
map.merge(word,
1L,
(x, y) -> x + y);
}
return map;
}
}
the compute() method constructs the
local word count
![Page 24: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/24.jpg)
Art of Multiprocessor Programming 24
WordCount MapperMap<String,Long> compute() {
Map<String,Long> map = new HashMap<>();
for (String word : input) {
map.merge(word,
1L,
(x, y) -> x + y);
}
return map;
}
}create a map to hold the output
![Page 25: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/25.jpg)
Art of Multiprocessor Programming 25
WordCount MapperMap<String,Long> compute() {
Map<String,Long> map = new HashMap<>();
for (String word : input) {
map.merge(word,
1L,
(x, y) -> x + y);
}
return map;
}
}
examine each word in
the document fragment
![Page 26: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/26.jpg)
Art of Multiprocessor Programming 26
WordCount MapperMap<String,Long> compute() {
Map<String,Long> map = new HashMap<>();
for (String word : input) {
map.merge(word,
1L,
(x, y) -> x + y);
}
return map;
}
} increment that word’s
count in the map
![Page 27: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/27.jpg)
Art of Multiprocessor Programming 27
WordCount MapperMap<String,Long> compute() {
Map<String,Long> map = new HashMap<>();
for (String word : input) {
map.merge(word,
1L,
(x, y) -> x + y);
}
return map;
}
}
when the local count is
complete, return the map
![Page 28: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/28.jpg)
Reduce Phase
Art of Multiprocessor Programming 28
…
alpha → 4
bravo → 2
…
zulu → 1
a reducer thread merges mapper outputs
alpha → 2
juliet → 1
tango → 1
…
alpha → 1
foxtrot → 1
papa→ 1
tango → 1
…
alpha → 1
oscar → 1,
bravo → 2…
![Page 29: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/29.jpg)
alpha → 3
bravo → 2
…
zulu → 1
Reduce Phase
Art of Multiprocessor Programming 29
the reducer task produces a stream …of key-value pairs …
key: word value: word count
![Page 30: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/30.jpg)
Art of Multiprocessor Programming 30
Reducer Classabstract class Reducer<K, V, OUT>
extends RecursiveTask<OUT> {
K key;
List<V> valueList;
public void setInput(
K aKey,
List<V> aList) {
key = aKey;
valueList = aList;
}
}
![Page 31: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/31.jpg)
Art of Multiprocessor Programming 31
Reducer Classabstract class Reducer<K, V, OUT>
extends RecursiveTask<OUT> {
K key;
List<V> valueList;
public void setInput(
K aKey,
List<V> aList) {
key = aKey;
valueList = aList;
}
}
each reducer is given
a single key (word)
![Page 32: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/32.jpg)
Art of Multiprocessor Programming 32
Reducer Classabstract class Reducer<K, V, OUT>
extends RecursiveTask<OUT> {
K key;
List<V> valueList;
public void setInput(
K aKey,
List<V> aList) {
key = aKey;
valueList = aList;
}
}
and a list of associated values
(word count per fragment)
![Page 33: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/33.jpg)
Art of Multiprocessor Programming 33
Reducer Classabstract class Reducer<K, V, OUT>
extends RecursiveTask<OUT> {
K key;
List<V> valueList;
public void setInput(
K aKey,
List<V> aList) {
key = aKey;
valueList = aList;
}
}
It produces a single summary value
(the total count for that word)
![Page 34: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/34.jpg)
WordCount
Art of Multiprocessor Programming 34
0.037
normalizing document wordcountgives a fingerprint vector
0.002
0.045
…
0.000
![Page 35: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/35.jpg)
Document Fingerprint
Art of Multiprocessor Programming 35
a fingerprint is a point in a
high-dimensional space
![Page 36: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/36.jpg)
Clustering
Art of Multiprocessor Programming 36
romance novels
tango lyrics
Usenix Procedings
![Page 37: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/37.jpg)
k-means
Art of Multiprocessor Programming 37
Find k clusters from raw data
![Page 38: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/38.jpg)
k-means
Art of Multiprocessor Programming 38
Find k clusters from raw data
each vector closer to those in same cluster …
than in different clusters.
![Page 39: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/39.jpg)
MapReduce
Art of Multiprocessor Programming 39
…
thread 1 thread 2 thread k
split points among mapping threads
![Page 40: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/40.jpg)
k-means
Art of Multiprocessor Programming 40
Reducer picks k “centers” at random
![Page 41: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/41.jpg)
Reduce Phase
Art of Multiprocessor Programming 41
0 → c0
1 → c1
2 → c2
…
thread 1 thread 2 thread k
reducer sends key-value pair to mappers
key: cluster number
value: center point
![Page 42: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/42.jpg)
Mappers
Art of Multiprocessor Programming 42
Each mapper uses
centers to assign each
vector to a cluster
0 → c0
1 → c1
2 → c2
![Page 43: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/43.jpg)
Mappers
Art of Multiprocessor Programming 43
Each mapper uses
centers to assign each
vector to a cluster
0 → c0
1 → c1
2 → c2
![Page 44: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/44.jpg)
Mappers
Art of Multiprocessor Programming 44
p0 → 2
p1 → 1
p2 → 1
p3 → 0
mapper sends key-value stream to reducer
key: point
value: cluster ID
![Page 45: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/45.jpg)
C0 = {…}
C1 = {…}
C2 = {…}
Back at the Reducer
Art of Multiprocessor Programming 45
The reducer merges the streams ….
and assembles clusters …
![Page 46: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/46.jpg)
Back at the Reducer
Art of Multiprocessor Programming 46
The reducer computes new centers
based on new clusters …
0 → c0’
1 → c1’
2 → c2’
![Page 47: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/47.jpg)
Once is Not Enough
Art of Multiprocessor Programming 47
…
thread 1 thread 2 thread k
reducer sends new centers to mappers
process ends when centers become stable
0 → c0’
1 → c1’
2 → c2’
![Page 48: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/48.jpg)
To Recaptulate
Art of Multiprocessor Programming 48
We saw two problems …
wordcount & k-means …
with similar parallel solutions
Map part is parallel …
Reduce part is sequential.
![Page 49: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/49.jpg)
Art of Multiprocessor Programming 49
abstraction
![Page 50: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/50.jpg)
Map Function
Art of Multiprocessor Programming 50
(k1, v1) list(k2, v2)
doc, contents word, count
cluster ID, center point, cluster ID
![Page 51: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/51.jpg)
Reduce Function
Art of Multiprocessor Programming 51
(k2, list(v2)) list(v2)
word, list of counts count
cluster ID, list of points new cluster center
![Page 52: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/52.jpg)
Example
Art of Multiprocessor Programming 52
Distributed Grep
Map:
line of document
Reduce:
copy line to display
![Page 53: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/53.jpg)
Example
Art of Multiprocessor Programming 53
URL Access Frequency
Map:
(URL, local count)
Reduce:
(URL, total count)
![Page 54: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/54.jpg)
Example
Art of Multiprocessor Programming 54
Reverse web link graph
Map:
(target link, source page)
Reduce:
(target link, list of source pages)
![Page 55: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/55.jpg)
Other Examples
Art of Multiprocessor Programming 55
histogram
matrix multiplication
PageRank
Betweenness centrality
![Page 56: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/56.jpg)
Art of Multiprocessor Programming 56
Distributed MapReduce
Google, Hadoop, etc…
Communication by message
Fault-tolerance important
![Page 57: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/57.jpg)
Art of Multiprocessor Programming 57
Multicore MapReduce
Phoenix, Phoenix++, Metis …
Communication by shared memory objects
Fault-tolerance unimportant
![Page 58: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/58.jpg)
Costs
Art of Multiprocessor Programming 58
key-value layout
memory allocation
mechanism overhead
cache pressure
static vs dynamic
![Page 59: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/59.jpg)
Part Two
Data Streams59
![Page 60: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/60.jpg)
Streams
Art of Multiprocessor Programming 60
sequence of
transformations
transformation
transformation
data
source
data
data
consumer
no relation to
I/O streams
sometimes in parallel
![Page 61: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/61.jpg)
Streams
Art of Multiprocessor Programming 61
transformation
transformation
data
source
data
data
consumer
transformations given by
mathematical functions
![Page 62: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/62.jpg)
Streams
Art of Multiprocessor Programming 62
transformation
transformation
data
source
data
data
consumer
transformation creates
new stream
no modifications or
side-effects
correctness easier?
![Page 63: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/63.jpg)
Functional Programming
Art of Multiprocessor Programming 63
functions map old state to new state
old state never changed
no complex side-effects
elegant, easier proofs of correctness
![Page 64: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/64.jpg)
Oh, Really?
Art of Multiprocessor Programming 64
“Functional languages are unnatural to
use; but so are knives and forks,
diplomatic protocols, double-entry
bookkeeping, and a host of other things
modern civilization has found useful.”
Jim Morris, 1982
![Page 65: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/65.jpg)
Haiku
Art of Multiprocessor Programming 65
esthetically pleasing
only works on those who understand Haiku
![Page 66: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/66.jpg)
Karate
Art of Multiprocessor Programming 66
esthetically pleasing
works even on those who do not understand Karate
![Page 67: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/67.jpg)
Jim Morris’s Question
Art of Multiprocessor Programming 67
Is functional programming more like Haiku or Karate?
1981: Haiku Today: Karate
![Page 68: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/68.jpg)
Laziness
Art of Multiprocessor Programming 68
x → x+1
1,2,3,…
x → 2x
sum
No computation until
absolutely necessary
add 1 to each element:
no computation
double each element: no
computation
2(xi+1) is terminal
![Page 69: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/69.jpg)
Laziness
Art of Multiprocessor Programming 69
x → x+1
1,2,3,…
x → 2x
collect in List
move to container
is terminal
![Page 70: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/70.jpg)
Laziness
Art of Multiprocessor Programming 70
x → x+1
1,2,3,…
x → 2x
collect in ListLaziness permits
optimizations
x → 2(x+1)
![Page 71: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/71.jpg)
Laziness
Art of Multiprocessor Programming 71
1 1 2 3 5 8 12 20 32 …
Laziness permits infinite streams …
Stream<Integer> fib = new FibStream();
![Page 72: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/72.jpg)
Art of Multiprocessor Programming 72
Unbounded Random Stream
Stream<Double> randomDoubleStream() {
return Stream.generate(
() -> random.nextDouble()
);
}
![Page 73: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/73.jpg)
Art of Multiprocessor Programming 73
Unbounded Random Stream
Stream<Double> randomDoubleStream() {
return Stream.generate(
() -> random.nextDouble()
);
}Unbounded stream of double-
precision random numbers
![Page 74: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/74.jpg)
Art of Multiprocessor Programming 74
Random Stream
Stream<Double> randomDoubleStream() {
return Stream.generate(
() -> random.nextDouble()
);
}Stream that generates new
elements on the fly
![Page 75: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/75.jpg)
Art of Multiprocessor Programming 75
Random Stream
Stream<Double> randomDoubleStream() {
return Stream.generate(
() -> random.nextDouble()
);
}function to call when
generating new element
example of Java lambda
expression (anon method)
![Page 76: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/76.jpg)
Art of Multiprocessor Programming 76
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
![Page 77: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/77.jpg)
Art of Multiprocessor Programming 77
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
puts each word from the document into a List
![Page 78: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/78.jpg)
Art of Multiprocessor Programming 78
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
open the file, create a FileReader
![Page 79: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/79.jpg)
Art of Multiprocessor Programming 79
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
how a stream program looks
each line creates a new stream
![Page 80: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/80.jpg)
Art of Multiprocessor Programming 80
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
turn the FileReader into a stream
of lines, each line a string
![Page 81: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/81.jpg)
Art of Multiprocessor Programming 81
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
map creates a new stream by applying a
function to each stream element
here, converts to lower case
![Page 82: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/82.jpg)
Art of Multiprocessor Programming 82
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
flatMap replaces one stream element
with multiple stream elements
here, splits line into words
![Page 83: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/83.jpg)
Art of Multiprocessor Programming 83
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
collect puts stream elements in a container
here, in a List
terminal operation
![Page 84: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/84.jpg)
Art of Multiprocessor Programming 84
WordCountList<String> readFile(String fileName) {
…
return reader
.lines()
.map(String::toLowerCase)
.flatMap(s ->
pattern.splitAsStream(s))
.collect(Collectors.toList());
}
No loops
No conditionals
No mutable objects
![Page 85: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/85.jpg)
Art of Multiprocessor Programming 85
WordCountMap<String,Long> map = text
.stream()
.collect(
Collectors.groupingBy(
Function.identity(),
Collectors.counting()));
now let’s count the words
![Page 86: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/86.jpg)
Art of Multiprocessor Programming 86
WordCountMap<String,Long> map = text
.stream()
.collect(
Collectors.groupingBy(
Function.identity(),
Collectors.counting()));
start with list of words
![Page 87: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/87.jpg)
Art of Multiprocessor Programming 87
WordCountMap<String,Long> map = text
.stream()
.collect(
Collectors.groupingBy(
Function.identity(),
Collectors.counting()));
turn List into a stream
![Page 88: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/88.jpg)
Art of Multiprocessor Programming 88
WordCountMap<String,Long> map = text
.stream()
.collect(
Collectors.groupingBy(
Function.identity(),
Collectors.counting()));
put stream into a container (Map)
word → count
![Page 89: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/89.jpg)
Art of Multiprocessor Programming 89
WordCountMap<String,Long> map = text
.stream()
.collect(
Collectors.groupingBy(
Function.identity(),
Collectors.counting()));
each element’s key is that element
![Page 90: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/90.jpg)
Art of Multiprocessor Programming 90
WordCountMap<String,Long> map = text
.stream()
.collect(
Collectors.groupingBy(
Function.identity(),
Collectors.counting()));
each element’s value is the number of
times it appears
![Page 91: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/91.jpg)
Art of Multiprocessor Programming 91
WordCountMap<String,Long> map = text
.stream()
.collect(
Collectors.groupingBy(
Function.identity(),
Collectors.counting()));
No loops
No conditionals
No mutable objects
![Page 92: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/92.jpg)
k-Means
Art of Multiprocessor Programming 92
class Point {
Point(double x, double y) {…}
Point plus(Point other) {…}
Point scale(double x) {…}
static Point barycenter(
List<Point> cluster
) {…}
}
![Page 93: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/93.jpg)
k-Means
Art of Multiprocessor Programming 93
class Point {
Point(double x, double y) {…}
Point plus(Point other) {…}
Point scale(double x) {…}
static Point barycenter(
List<Point> cluster
) {…}
}
stream-based!
![Page 94: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/94.jpg)
BaryCenter
Art of Multiprocessor Programming 94
![Page 95: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/95.jpg)
Stream Barycenter
Art of Multiprocessor Programming 95
Point barycenter(List<Point> cluster){
double numPoints = cluster.size();
Optional<Point> sum = cluster
.stream()
.reduce(Point::plus);
return sum.get()
.scale(1 / numPoints);
}
![Page 96: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/96.jpg)
Stream Barycenter
Art of Multiprocessor Programming 96
Point barycenter(List<Point> cluster){
double numPoints = cluster.size();
Optional<Point> sum = cluster
.stream()
.reduce(Point::plus);
return sum.get()
.scale(1 / numPoints);
} cluster size
![Page 97: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/97.jpg)
Stream Barycenter
Art of Multiprocessor Programming 97
Point barycenter(List<Point> cluster){
double numPoints = cluster.size();
Optional<Point> sum = cluster
.stream()
.reduce(Point::plus);
return sum.get()
.scale(1 / numPoints);
}turn List into Stream
![Page 98: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/98.jpg)
Reduce
Art of Multiprocessor Programming 98
stream.reduce(+):
() → ;
(a) → a
(a,b) → a+b
(a,b,c) → (a+b)+c
etc. …terminal operation
![Page 99: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/99.jpg)
k-Means
Art of Multiprocessor Programming 99
Point barycenter(List<Point> cluster){
double numPoints = cluster.size();
Optional<Point> sum = cluster
.stream()
.reduce(Point::plus);
return sum.get()
.scale(1 / numPoints);
} sum points in cluster
Optional because sum might be empty!
![Page 100: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/100.jpg)
k-Means
Art of Multiprocessor Programming 100
Point barycenter(List<Point> cluster){
double numPoints = cluster.size();
Optional<Point> sum = cluster
.stream()
.reduce(Point::plus);
return sum.get()
.scale(1 / numPoints);
}
extract sum and divide by # points
![Page 101: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/101.jpg)
k-Means
Art of Multiprocessor Programming 101
List<Point> points =
readFile("cluster.dat");
centers =
randomDistinctCenters(points);
double convergence = 1.0;
while (convergence > EPSILON) {
…
}
![Page 102: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/102.jpg)
k-Means
Art of Multiprocessor Programming 102
List<Point> points =
readFile("cluster.dat");
centers =
randomDistinctCenters(points);
double convergence = 1.0;
while (convergence > EPSILON) {
…
}
read points from file
![Page 103: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/103.jpg)
k-Means
Art of Multiprocessor Programming 103
List<Point> points =
readFile("cluster.dat");
centers =
randomDistinctCenters(points);
double convergence = 1.0;
while (convergence > EPSILON) {
…
}pick random centers
![Page 104: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/104.jpg)
k-Means
Art of Multiprocessor Programming 104
List<Point> points =
readFile("cluster.dat");
centers =
randomDistinctCenters(points);
double convergence = 1.0;
while (convergence > EPSILON) {
…
}
keep going until centers are stable
![Page 105: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/105.jpg)
Compute New Clusters
Art of Multiprocessor Programming 105
while (convergence > EPSILON) {
Map<Integer,List<Point>> clusters
= points
.stream()
.collect(
Collectors.groupingBy(
p ->
closestCenter(centers, p)
)
);
}
turn list of points into a Stream
![Page 106: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/106.jpg)
Compute New Clusters
Art of Multiprocessor Programming 106
while (convergence > EPSILON) {
Map<Integer,List<Point>> clusters
= points
.stream()
.collect(
Collectors.groupingBy(
p ->
closestCenter(centers, p)
)
);
}
put each point in a map
key is closest center
value is list of points with that center
![Page 107: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/107.jpg)
k-Means Con’t
Art of Multiprocessor Programming 107
while (convergence > EPSILON) {
...
Map<Integer, Point> newCenters
= clusters
.entrySet()
.stream()
.collect(
Collectors.toMap(
e -> e.getKey(),
e -> Point.barycenter(e.getValue())
)
);
convergence = distance(centers, newCenters);
centers = newCenters;
}
compute new centers
map: ID → center
![Page 108: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/108.jpg)
k-Means Con’t
Art of Multiprocessor Programming 108
while (convergence > EPSILON) {
...
Map<Integer, Point> newCenters
= clusters
.entrySet()
.stream()
.collect(
Collectors.toMap(
e -> e.getKey(),
e -> Point.barycenter(e.getValue())
)
);
convergence = distance(centers, newCenters);
centers = newCenters;
}
turn map into a Stream of
(ID, point) pairs
![Page 109: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/109.jpg)
k-Means Con’t
Art of Multiprocessor Programming 109
while (convergence > EPSILON) {
...
Map<Integer, Point> newCenters
= clusters
.entrySet()
.stream()
.collect(
Collectors.toMap(
e -> e.getKey(),
e -> Point.barycenter(e.getValue())
)
);
convergence = distance(centers, newCenters);
centers = newCenters;
}
turn stream into a map
ID → barycenter
![Page 110: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/110.jpg)
k-Means Con’t
Art of Multiprocessor Programming 110
while (convergence > EPSILON) {
...
Map<Integer, Point> newCenters
= clusters
.entrySet()
.stream()
.collect(
Collectors.toMap(
e -> e.getKey(),
e -> Point.barycenter(e.getValue())
)
);
convergence = distance(centers, newCenters);
centers = newCenters;
}
new Key is still the cluster ID
![Page 111: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/111.jpg)
k-Means Con’t
Art of Multiprocessor Programming 111
while (convergence > EPSILON) {
...
Map<Integer, Point> newCenters
= clusters
.entrySet()
.stream()
.collect(
Collectors.toMap(
e -> e.getKey(),
e -> Point.barycenter(e.getValue())
)
);
convergence = distance(centers, newCenters);
centers = newCenters;
}
new value is the barycenter computed earlier
![Page 112: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/112.jpg)
k-Means Con’t
Art of Multiprocessor Programming 112
while (convergence > EPSILON) {
...
Map<Integer, Point> newCenters
= clusters
.entrySet()
.stream()
.collect(
Collectors.toMap(
e -> e.getKey(),
e -> Point.barycenter(e.getValue())
)
);
convergence = distance(centers, newCenters);
centers = newCenters;
}
If centers have moved, start
again with the new centers
![Page 113: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/113.jpg)
Functional k-Means
Art of Multiprocessor Programming 113
many fewer lines of code
easier to read (really!)
easier to reason
easier to optimize
![Page 114: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/114.jpg)
Art of Multiprocessor Programming 114
Parallelism?Arrays.asList("Arlington",
"Berkeley",
"Clarendon",
"Dartmouth",
"Exeter")
.stream()
.forEach(s -> printf("%s\n", s));
So far, Streams are sequential
![Page 115: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/115.jpg)
Art of Multiprocessor Programming 115
Parallelism?Arrays.asList("Arlington",
"Berkeley",
"Clarendon",
"Dartmouth",
"Exeter")
.stream()
.forEach(s -> printf("%s\n", s));
make List of strings and turn them into a stream
![Page 116: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/116.jpg)
Art of Multiprocessor Programming 116
Parallelism?Arrays.asList("Arlington",
"Berkeley",
"Clarendon",
"Dartmouth",
"Exeter")
.stream()
.forEach(s -> printf("%s\n", s));
forEach applies a method to
each element (not functional)
![Page 117: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/117.jpg)
Art of Multiprocessor Programming 117
Parallelism?Arrays.asList("Arlington",
"Berkeley",
"Clarendon",
"Dartmouth",
"Exeter")
.stream()
.forEach(s -> printf("%s\n", s));Arlington
Berkeley
Clarendon
Dartmouth
Exeter
prints
![Page 118: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/118.jpg)
Art of Multiprocessor Programming 118
Parallelism?Arrays.asList("Arlington",
"Berkeley",
"Clarendon",
"Dartmouth",
"Exeter")
.parallelStream()
.forEach(s -> printf("%s\n", s));
turn List into a parallel stream
![Page 119: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/119.jpg)
Art of Multiprocessor Programming 119
Parallelism?Arrays.asList("Arlington",
"Berkeley",
"Clarendon",
"Dartmouth",
"Exeter")
.parallelStream()
.forEach(s -> printf("%s\n", s));Arlington
Berkeley
Clarendon
Dartmouth
Exeter
prints
Dartmouth
Arlington
Clarendon
Berkeley
Exeter
Dartmouth
Berkeley
Exeter
Arlington
Clarendon
![Page 120: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/120.jpg)
Art of Multiprocessor Programming 120
Parallelism?Arrays.asList("Arlington",
"Berkeley",
"Clarendon",
"Dartmouth",
"Exeter")
.stream()
.parallel()
.forEach(s -> printf("%s\n", s));
can turn stream into a parallel stream
![Page 121: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/121.jpg)
Art of Multiprocessor Programming 121
Pitfallslist.stream().forEach(
s -> list.add(0)
);
![Page 122: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/122.jpg)
Art of Multiprocessor Programming 122
Pitfallslist.stream().forEach(
s -> list.add(0)
);
lambda (function) must not modify source!
![Page 123: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/123.jpg)
Art of Multiprocessor Programming 123
Pitfallssource.parallelStream()
.forEach(
s -> target.add(s));
exception if target not thread-safeorder added is non-deterministic
![Page 124: Art of Multiprocessor Programming](https://reader030.fdocuments.us/reader030/viewer/2022012523/619703d6af8a254b716f1d29/html5/thumbnails/124.jpg)
Conclusions
Art of Multiprocessor Programming 124
Streams support
functional programming
data parallelism
compiler optimizations