Hot Streaming Java

1

Hot Streaming Java

Nick MaioranoThoughtFlow Solutions Inc.

Java Consultant

Author

Trainer

2Hot Streaming Java

About me

3Hot Streaming Java

About me

Goals

4

• Understand the streams API• Able to use them in everyday Java• Know when to hold ‘em/fold ‘em

Hot Streaming Java

Objectives

5

• Review the functional paradigm• Present stream operations• Examine stream-based algorithms• Compare imperative algorithms vs.

serial & parallel streams

Hot Streaming Java

Part I

Introduction

6Hot Streaming Java

Streams

7

• Introduced as part of Java 8• Minimal syntactical changes in Java 8• Streams are as functional as Java gets

Hot Streaming Java

What’s a stream?

8

“A declarative construct used to express an algorithm as a series of operations working

on a stream of data”

Hot Streaming Java

9Hot Streaming Java

What’s a stream?

10Hot Streaming Java

What’s a stream?

Streams


• Streams are functional constructs:• Functions as first-class citizens• Emphasis on immutability/

statelessness• Avoidance of side-effects• Declarative programming

Streams


• Streams use:• Lambdas• Functional interfaces• Method references

Lambda refresh


Lambda expressionsParameter name -> Lambda expression

i -> System.out.println(i);

i -> i * 2;

Function <T, R>R apply(T t);

Supplier<T>T get()

Functional Interfaces


Functional

Interfaces

Consumer

Function

Predicate

Supplier

Consumer<T> void accept(T t);

Predicate<T>boolean test(T t);

Functional Interfaces


BiFunction<Integer, Integer, Integer>

multiply = (x, y) -> x * y;

IntBinaryOperator multiply2 = (x, y) -> x * y;

IntBinaryOperator multiply3 = Math::multiplyExact;

int product = multiply.apply(10, 20);

int product2 = multiply2.applyAsInt(10, 20);

int product3 = multiply3.applyAsInt(10, 20);


Streams

Lambda Lambda Lambda Lambda

Operation 1

Operation 2

Operation 3

Operation 4

• Operations fused into pipelines• Customized with lambdas• Abstract for/while loop• Fluent-style programming


Streams

List<Integer> integersIn =

Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10);

List<Integer> evenIntegers = new ArrayList<>();

integersIn.stream().

filter(x -> x % 2 == 0).

forEach(evenIntegers::add);

parallel().


Constructing streams

A coherent stream

1 build + 0..n intermediate + 1 terminal



Good stream

list.stream().filter(x-> x ^ 2 == 0).reduce((l, r) -> l + r);

Build operation Intermediate operation Terminal operation



Bad stream!

list.stream().filter(x-> x ^ 2 == 0);

No terminal operation



Badder stream!!

Stream.iterate(0, i -> ++i).anyMatch(i -> i < 0);

Infinite stream



Baddest stream!!!

List<String> words = new ArrayList<>(

Arrays.asList(“This”, “sentence”, “contains”, “five”, “words”));

// This stream has interference – avoid!

words.stream().

forEach(s -> {if (s.equals(“five”)) words.add(“thousand”);});

Lambda interferes with stream, has side-effects and is not thread-safe



Generic vs. specialized

//Generic

Stream.iterate(1, i -> ++i).

limit(10).reduce((l, r) -> l + r);

//Specialized

IntStream.rangeClosed(1, 10).sum();

Part IIDiving into Streams


Stream operations


Stream

operations

Build

Filter

Map

Reduce

Iterate

Peek

• Create the stream• Collections:

List.stream()

• Buffered readers: BufferedReader.lines()

• Or thin-air:IntStream.range(1, 10)

Stream operations


Stream

operations

Build

Filter

Map

Reduce

Iterate

Peek

• Allows or blocks data• If-statements• True:

• let the stream element thru

• False: • block the element

Stream operations


Stream

operations

Build

Filter

Map

Reduce

Iterate

Peek

• Transforms stream data• Same or different type

• map, mapToInt, mapToLong

• Same or different cardinality• flatMap, flatMapToInt, flatMapToLong

Stream operations


Stream

operations

Build

Filter

Map

Reduce

Iterate

Peek

• Boils down to one thing• Ordinary reduction:

(((n1 + n2) + n3) + n4) + n5

(((1 + 2) + 3) + 4) + 5((3 + 3) + 4) + 5

(6 + 4) + 510 + 5

15

• Mutable reduction:• Mutate an object while collecting

Left fold


Implement the Linux grep command using streams to count string matches


Stream grep –cImperative algorithm

private static long grepDashCImperative(BufferedReader in,

String upperCaseSearchWord) {

String nextLine = in.readLine();

int count = 0;

while (nextLine != null) {

String upperCaseLine = nextLine.toUpperCase();

if (upperCaseLine.contains(upperCaseSearchWord)) {

count++;

}

nextLine = in.readLine();

}

return count;

}

Imperative style


Stream grep –cUsing forEach

private static long grepDashC(BufferedReader in,


int count = 0;

in.lines().

map(String::toUpperCase).

filter(s -> s.contains(upperCaseSearchWord)).

forEach(next -> count++);

return count;

}

Stream style


Stream grep –cUsing forEach



int count = 0;

in.lines().



forEach(next -> count++);

return count;

} Cannot mutate a local variable in a lambda


Stream grep –cUsing reduce



// The long way

return in.lines().



mapToLong(count -> 1).

reduce(0, (l, r) -> l + r);

}

Using the reduce

operation


Stream grep –cUsing count()



// Using the built-in function count

return in.lines().



count();

}

Using the count

operation


Implement the Linux grep command using streams to return string matches


Stream grepForeach-based accumulation

private static String grep(BufferedReader in,


StringBuilder accumulator = new StringBuilder();

// Accumulate the strings via foreach

in.lines().



forEach(accumulator::append);

return accumulator.toString();

}

forEach is not a reduction tool


Stream grepReduction-based accumulation

private static String grep(BufferedReader in,


// Accumulate the strings via reduction

return in.lines().



reduce("", (l, r) -> l.concat(r).concat(", "));

}

Reduce is not the right tool for the job:String is being copied every time


Stream grepMutable reduction

private static List<String> grep(BufferedReader in,

String upperCaseSearchWord){

// Accumulate the strings via collect

return in.lines().



collect(ArrayList<String>::new,

ArrayList<String>::add,

ArrayList<String>::addAll);

}

Collect is a mutable reduction operation

Collect

Supplier(Supplier)

Creates the mutable structure

Accumulator(BiConsumer)

Adds to the structure

Combiner(BiConsumer)

Merges each partial

accumulation

ArrayList<String>::add

ArrayList<String>::addAllArrayList<String>::new





// Accumulate the strings via collect

return in.lines().



collect(ArrayList<String>::new,

ArrayList<String>::add,

ArrayList<String>::addAll);

}

Scary syntax can be replaced by…





// Accumulate the strings via collect with collectors

return in.lines().



collect(Collectors.toList());

}

Declarative collection

Rolling your own Collector


• Implement Collector interface• Supplier, Accumulator, Combiner, Finisher

• Set characteristics• Concurrent: accumulation can be concurrent• Unordered: Collection is not ordered• Identity finish: Turn on/off finishing


Stream grepParallel streaming



// Accumulate the strings via collect with collectors

return in.lines().

map(String::toUpperCase).parallel().


collect(Collectors.toList());

}


Serial streaming


Parallel streaming

651

82

9743

974382651

Part IIIFluid Streams


Stream thinking

47

• Thinking in streams can be difficult• Steep learning curve

• Lambdas• Method references• Standard functional interfaces• Generics

• Requires a functional mindset• Look at algorithms differently• Not all algorithms translate to streams

Hot Streaming Java

Stream thinking

Streams are best suited for algorithms

that… Traverse a sequence of data

Reduces sequence to a thing

No read-ahead no read-behind

No state change during traversal

Stream thinking

49

• When algorithms don’t fit:• Re-think the algorithm functionally• Use stream operations to maintain state• Or don’t use streams!

Hot Streaming Java

Algorithm suitability

Grep

Quick sort

Conway’s game of life


Compare imperative quick sort vs.

stream quick sort


public static ArrayList<Integer> imperativeQuickSort(ArrayList<Integer> array, int low, int n) {int lo = low;int hi = n;

if (lo >= n) return array;

// Step 1: find pivot pointint mid = array.get((lo + hi) / 2);

// Step 2: find & swap values less & greater than pivotwhile (lo < hi) {

while (lo < hi && array.get(lo) < mid) {lo++;

}while (lo < hi && array.get(hi) > mid) {

hi--;}if (lo < hi) {

// Swap valuesint temp = array.get(lo);array.set(lo, array.get(hi));array.set(hi, temp);lo++;hi--;

}}

if (hi < lo) lo = hi;

// Steps 3: split the array and repeat recursivelyimperativeQuickSort(array, low, lo);imperativeQuickSort(array, lo == low ? lo + 1 : lo, n);

Imperative quick sort

public static List<Integer> functionalSort(List<Integer> array) {

List<Integer> returnArray = array;

if (array.size() > 1) {

// Step 1

int mid = array.get(array.size() / 2);

// Step 2

Map<Integer, List<Integer>> map = array.stream().parallel().

collect(Collectors.groupingBy(i -> i < mid ? 0 : i == mid ? 1 : 2));

// Step 3

List<Integer> left = functionalSort(map.getOrDefault(0, new ArrayList<>()));

List<Integer> middle = map.getOrDefault(1, new ArrayList<>());

List<Integer> right = functionalSort(map.getOrDefault(2, new ArrayList<>()));

left.addAll(middle);

left.addAll(right);

returnArray = left;

}

return returnArray;

}


Functional quick sort


Compare imperative Conway’s Game Of Lifevs.

stream-based Game of life


public static boolean[][] getNextGeneration(boolean[][] oldGeneration)

{

boolean[][] newGeneration = new boolean[oldGeneration.length][oldGeneration.length];

for (int y = 0; y < oldGeneration.length; ++y) {

for (int x = 0; x < oldGeneration.length; ++x) {

newGeneration[y][x] = isAlive(oldGeneration, y, x);

}

}

return newGeneration;

}

private static int countLiveNeighborCells(boolean[][] generation, int y, int x)

{

int count = 0;

for (int yIndex = y - 1; yIndex <= y + 1; ++yIndex) {

if (yIndex >= 0 && yIndex < generation.length) {

for (int xIndex = x - 1; xIndex <= x + 1; ++xIndex) {

if (xIndex >= 0 && xIndex < generation.length) {

if (generation[yIndex][xIndex] && !(xIndex == x && yIndex == y)) {

++count;

}

}

}

}

}

return count;

}

private static boolean isAlive(boolean[][] generation, int y, int x)

{

int liveCells = countLiveNeighborCells(generation, y, x);

return (generation[y][x] && liveCells >= 2 && liveCells <= 3) ||

(!generation[y][x] && liveCells == 3);

}

Imp

erat

ive


Conway’s Game of Lifepublic static boolean[][] getNextGeneration(boolean[][] oldGeneration)

{


for (int y = 0; y < oldGeneration.length; ++y) {

for (int x = 0; x < oldGeneration.length; ++x) {

newGeneration[y][x] = isAlive(oldGeneration, y, x);

}

}


}


{

int count = 0;

for (int yIndex = y - 1; yIndex <= y + 1; ++yIndex) {

if (yIndex >= 0 && yIndex < generation.length) {

for (int xIndex = x - 1; xIndex <= x + 1; ++xIndex) {

if (xIndex >= 0 && xIndex < generation.length) {

if (generation[yIndex][xIndex] && !(xIndex == x && yIndex == y)) {

++count;

}

}

}

}

}

return count;

}

private static boolean isAlive(boolean[][] generation, int y, int x)

{

int liveCells = countLiveNeighborCells(generation, y, x);

return (generation[y][x] && liveCells >= 2 && liveCells <= 3) ||

(!generation[y][x] && liveCells == 3);

}

public static boolean[][] getNextGenerationFunctional(boolean[][] oldGeneration)

{


List<Coordinates> list = new ArrayList<>();

IntStream.range(0, oldGeneration.length).

forEach(nextY ->

{

list.addAll(IntStream.range(0, oldGeneration.length).

parallel().

mapToObj(nextX -> new Coordinates(nextX, nextY)).

filter(coordinates -> isAlive(oldGeneration, coordinates.getY(),

coordinates.getX())).

collect(Collectors.toList()));

});

list.parallelStream().

forEach(nextCoordinate ->

newGeneration[nextCoordinate.getY()][nextCoordinate.getX()] = true);


}


{

return IntStream.rangeClosed(y - 1, y + 1).

filter(yInner -> yInner >= 0 && yInner < generation.length).

reduce(0, (yLeft, yRight) -> yLeft +

IntStream.rangeClosed(x - 1, x + 1).

filter(xInner -> xInner >= 0 && xInner < generation.length &&

generation[yRight][xInner] && !(xInner == x && yRight == y)).

reduce(0, (xLeft, xRight) -> xLeft + 1));

}

Stre

am-b

ased

Takeaway

Grep & Quick Sort offered compelling reasons to implement with Streams

Conway’s Game of Life did not


Part IVParallel streams

Parallel streams

Parallel streams are best suited for

algorithms that… Can split data easily & quickly

Have noorder

constraint

Can reduce concurrently

Have large N and/or Q

Hot Streaming Java


Parallel streaming

Remember: A parallel stream must do everything a serial one does and more


Parallel streaming

Sometimes, parallel streams are slower than their serial counterparts…

And also their imperative counterparts

Part VEnd-of-stream


Streams recap

Hot Streaming Java

5

Think functionally when using streams

4

3

2

1

Use the right reduction tool

Break through the learning curve

Use parallel streaming intelligently

Adapt thinking for streams or don’t use ‘em

Questions


Hot Streaming Java

Software

Transcript of Hot Streaming Java