Lessons Learnt With Lambdas and Streams in JDK 8

© Copyright Azul Systems 2016 Agenda  Lambdas and Streams Primer  Delaying Execution  Avoiding Loops In Streams  The Art Of Reduction  Parallel streams: myths and realities  Lambdas and Streams and JDK 9  Conclusions 3

© Copyright Azul Systems 2016 Lambda Expressions In JDK 8  Old style, anonymous inner classes  New style, using a Lambda expression  Can be used wherever the type is a functional interface 5 Simplified Parameterised Behaviour new Thread(new Runnable { public void run() { doSomeStuff(); } }).start(); new Thread(() -> doSomeStuff()).start();

© Copyright Azul Systems 2016 Functional Interface Definition  Is an interface  Must have only one abstract method – In JDK 7 this would mean only one method (like ActionListener)  JDK 8 introduced default methods – Adding multiple inheritance of types to Java – These are, by definition, not abstract  JDK 8 also now allows interfaces to have static methods  @FunctionalInterface to have the compiler check 6

© Copyright Azul Systems 2016 Type Inference  Compiler can often infer parameter types in a lambda expression – Inferrence based on target functional interface’s method signature  Fully statically typed (no dynamic typing sneaking in) – More typing with less typing static T void sort(List<T> l, Comparator<? super T> c); List<String> list = getList(); Collections.sort(list, (String x, String y) -> x.length() > y.length()); Collections.sort(list, (x, y) -> x.length() - y.length());

© Copyright Azul Systems 2016 Stream Overview  A stream pipeline consists of three types of things – A source – Zero or more intermediate operations – A terminal operation  Producing a result or a side-effect int total = transactions.stream() .filter(t -> t.getBuyer().getCity().equals(“London”)) .mapToInt(Transaction::getPrice) .sum(); Source Intermediate operation Terminal operation

© Copyright Azul Systems 2016 Stream Terminal Operations  The pipeline is only evaluated when the terminal operation is called – All operations can execute sequentially or in parallel – Intermediate operations can be merged  Avoiding multiple redundant passes on data  Short-circuit operations (e.g. findFirst)  Lazy evaluation – Stream characteristics help identify optimisations  DISTINT stream passed to distinct() is a no-op

© Copyright Azul Systems 2016 Optional Class  Terminal operations like min(), max(), etc do not return a direct result – Suppose the input Stream is empty?  Optional<T> – Container for an object reference (null, or real object) – Think of it like a Stream of 0 or 1 elements – use get(), ifPresent() and orElse() to access the stored reference – Can use in more complex ways: filter(), map(), etc

© Copyright Azul Systems 2016 Performance Impact For Logging  Heisenberg’s uncertainty principle  Setting log level to INFO still has a performance impact  Since Logger determines whether to log the message the parameter must be evaluated even when not used 12 logger.finest(getSomeStatusData()); Always executed

© Copyright Azul Systems 2016 Supplier<T>  Represents a supplier of results  All relevant logging methods now have a version that takes a Supplier  Pass a description of how to create the log message – Not the message  If the Logger doesn’t need the value it doesn’t invoke the Lambda  Can be used for other conditional activities 13 logger.finest(getSomeStatusData());logger.finest(() -> getSomeStatusData());

© Copyright Azul Systems 2016 Functional v. Imperative  For functional programming you should not modify state  Java supports closures over values, not closures over variables  But state is really useful… 15

© Copyright Azul Systems 2016 Counting Methods That Return Streams 16 Still Thinking Imperatively Set<String> sourceKeySet = streamReturningMethodMap.keySet(); LongAdder sourceCount = new LongAdder(); sourceKeySet.stream() .forEach(c -> sourceCount .add(streamReturningMethodMap.get(c).size()));

© Copyright Azul Systems 2016 Printing And Counting 18 Still Thinking Imperatively LongAdder newMethodCount = new LongAdder(); functionalParameterMethodMap.get(c).stream() .forEach(m -> { output.println(m); if (isNewMethod(c, m)) newMethodCount.increment(); });

© Copyright Azul Systems 2016 Printing And Counting 19 More Functional, But Not Pure Functional int count = functionalParameterMethodMap.get(c).stream() .mapToInt(m -> { int newMethod = 0; output.println(m); if (isNewMethod(c, m)) newMethod = 1; return newMethod }) .sum(); There is still state being modified in the Lambda

© Copyright Azul Systems 2016 Printing And Counting 20 Even More Functional, But Still Not Pure Functional int count = functionalParameterMethodMap.get(nameOfClass) .stream() .peek(method -> output.println(method)) .mapToInt(m -> isNewMethod(nameOfClass, m) ? 1 : 0) .sum(); Strictly speaking printing is a side effect, which is not purely functional

© Copyright Azul Systems 2016 A Simple Problem  Find the length of the longest line in a file  Hint: BufferedReader has a new method, lines(), that returns a Stream 22 BufferedReader reader = ... int longest = reader.lines() .mapToInt(String::length) .max() .getAsInt();

© Copyright Azul Systems 2016 Naïve Stream Solution  That works, so job done, right?  Not really. Big files will take a long time and a lot of resources  Must be a better approach 25 String longest = reader.lines(). sort((x, y) -> y.length() - x.length()). findFirst(). get();

© Copyright Azul Systems 2016 External Iteration Solution  Simple, but inherently serial  Not thread safe due to mutable state 26 String longest = ""; while ((String s = reader.readLine()) != null) if (s.length() > longest.length()) longest = s;

© Copyright Azul Systems 2016 Recursive Approach 27 String findLongestString(String longest, BufferedReader reader) { String next = reader.readLine(); if (next == null) return longest; if (next.length() > longest.length()) longest = next; return findLongestString(longest, reader); }

© Copyright Azul Systems 2016 Recursion: Solving The Problem  No explicit loop, no mutable state, we’re all good now, right?  Unfortunately not: – larger data sets will generate an OOM exception – Too many stack frames 28 String longest = findLongestString("", reader);

© Copyright Azul Systems 2016 A Better Stream Solution  Stream API uses the well known filter-map-reduce pattern  For this problem we do not need to filter or map, just reduce Optional<T> reduce(BinaryOperator<T> accumulator)  BinaryOperator is a subclass of BiFunction – R apply(T t, U u)  For BinaryOperator all types are the same – T apply(T x, T y) 29

© Copyright Azul Systems 2016 A Better Stream Solution  The key is to find the right accumulator – The accumulator takes a partial result and the next element, and returns a new partial result – In essence it does the same as our recursive solution – But without all the stack frames 30

© Copyright Azul Systems 2016 A Better Stream Solution  Use the recursive approach as an accululator for a reduction 31 String longestLine = reader.lines() .reduce((x, y) -> { if (x.length() > y.length()) return x; return y; }) .get();

© Copyright Azul Systems 2016 A Better Stream Solution  Use the recursive approach as an accululator for a reduction 32 String longestLine = reader.lines() .reduce((x, y) -> { if (x.length() > y.length()) return x; return y; }) .get(); x in effect maintains state for us, by providing the partial result, which is the longest string found so far

© Copyright Azul Systems 2016 The Simplest Stream Solution  Use a specialised form of max()  One that takes a Comparator as a parameter  comparingInt() is a static method on Comparator Comparator<T> comparingInt( ToIntFunction<? extends T> keyExtractor) 33 reader.lines() .max(comparingInt(String::length)) .get();

© Copyright Azul Systems 2016 Serial And Parallel Streams  Syntactically very simple to change from serial to parallel 36 int sum = list.stream() .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum(); int sum = list.parallelStream() .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum();

© Copyright Azul Systems 2016 Serial And Parallel Streams  Syntactically very simple to change from serial to parallel  Use serial() or parallel() to change mid-stream 37 int sum = list.parallelStream() .filter(w -> w.getColour() == RED) .serial() .mapToInt(w -> w.getWeight()) .parallel() .sum();

© Copyright Azul Systems 2016 Serial And Parallel Streams  Syntactically very simple to change from serial to parallel  Use serial() or parallel() to change mid-stream  Whole stream is processed serially or in parallel – Last call wins  Operations should be stateless and independent 38 int sum = list.parallelStream() .filter(w -> w.getColour() == RED) .serial() .mapToInt(w -> w.getWeight()) .parallel() .sum();

© Copyright Azul Systems 2016 Common ForkJoinPool  Created when JVM starts up  Default number of threads is equal to CPUs reported by OS – Runtime.getRuntime().availableProcessors()  Can be changed manually 39 -Djava.util.concurrent.ForkJoinPool.common.parallelism=n

© Copyright Azul Systems 2016 Common ForkJoinPool 40 Set<String> workers = new ConcurrentSet<String>(); int sum = list.parallelStream() .peek(n -> workers.add(Thread.currentThread().getName())) .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum(); System.out.println(”Worker thread count = ” + workers.size()); -Djava.util.concurrent.ForkJoinPool.common.parallelism=4 Worker thread count = 5

© Copyright Azul Systems 2016 Common ForkJoinPool 41 Set<String> workers = new ConcurrentSet<String>(); ... worker.stream().forEach(System.out::println); -Djava.util.concurrent.ForkJoinPool.common.parallelism=4 ForkJoinPool.commonPool-worker-0 ForkJoinPool.commonPool-worker-1 ForkJoinPool.commonPool-worker-2 ForkJoinPool.commonPool-worker-3 main

© Copyright Azul Systems 2016 Parallel Stream Threads  Invoking thread is also used as worker  Parallel stream invokes ForkJoin synchronously – Blocks until work is finished  Other application threads using parallel stream will be affected  Beware IO blocking actions inside stream operartions  Do NOT nest parallel streams

© Copyright Azul Systems 2016 Parallel Stream Custom Pool Hack 43 ForkJoinPool customPool = new ForkJoinPool(4); ForkJoinTask<Integer> hackTask = customPool.submit(() -> { return list.parallelStream() .peek(w -> workers.add(Thread.currentThread().getName())) .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum(); }); int sum = hackTask.get(); Worker thread count = 4

© Copyright Azul Systems 2016 Is A Parallel Stream Faster?  Using a parallel stream guarantees more work – Setting up fork-join framework is overhead  This might complete more quickly  Depends on several factors – How many elements in the stream (N) – How long each element takes to process (T) – Parallel streams improve with N x T – Operation types  map(), filter() good  sort(), distinct() not so good 44

© Copyright Azul Systems 2016 Additional APIs  Optional now has a stream() method – Returns a stream of one element or an empty stream  Collectors.flatMapping() – Returns a Collector that converts a stream from one type to another by applying a flat mapping function 46

© Copyright Azul Systems 2016 Additional APIs  Matcher stream support – Stream<MatchResult> results()  Scanner stream support – Stream<MatchResult> findAll(String pattern) – Stream<MatchResult> findAll(Pattern pattern) – Stream<String> tokens() 47

© Copyright Azul Systems 2016 Additional Stream Sources  java.net.NetworkInterface – Stream<InetAddress> inetAddresses() – Stream<NetworkInterface> subInterfaces() – Stream<NetworkInterface> networkInterfaces()  static  java.security.PermissionCollection – Stream<Permission> elementsAsStream() 48

© Copyright Azul Systems 2016 Parallel Support For Files.lines()  Memory map file for UTF-8, ISO 8859-1, US-ASCII – Character sets where line feeds easily identifiable  Efficient splitting of mapped memory region  Divides approximately in half – To nearest line feed 49

© Copyright Azul Systems 2016 Stream takeWhile  Stream<T> takeWhile(Predicate<? super T> p)  Select elements from stream until Predicate matches  Unordered stream needs consideration thermalReader.lines() .mapToInt(i -> Integer.parseInt(i)) .takeWhile(i -> i < 56) .forEach(System.out::println);

© Copyright Azul Systems 2016 Stream dropWhile  Stream<T> dropWhile(Predicate<? super T> p)  Ignore elements from stream until Predicate matches  Unordered stream still needs consideration thermalReader.lines() .mapToInt(i -> Integer.parseInt(i)) .dropWhile(i -> i < 56) .forEach(System.out::println);

© Copyright Azul Systems 2016 Conclusions  Lambdas and Stream are a very powerful combination  Does require developers to think differently – Avoid loops, even non-obvious ones! – Reductions  Be careful with parallel streams  More to come in JDK 9 (and 10)  Join the Zulu.org community – www.zulu.org 54

Lessons Learnt With Lambdas and Streams in JDK 8

More Related Content

What's hot

Viewers also liked

Similar to Lessons Learnt With Lambdas and Streams in JDK 8

More from Simon Ritter

Recently uploaded

Lessons Learnt With Lambdas and Streams in JDK 8