DZone Java 8 Block Buster: Query Databases Using Streams

Meetup Gothenburg Dec 28 2015 Per-Åke Minborg minborg@speedment.com

Palo Alto Office Speedment Java One Presentation

Before Speedment Connector JDBC ORM JPA Application PROS Widely spread standard • Many providers • Many developers You can work with a relational database as if it were object oriented  Increased productivity  Provides a certain degree of abstraction (you can replace your DBMS) CONS  Slows down the application and gives unpredictable performance  You can not access all Java 8 features Time consuming to write applications for existing databases, You must still write SQL, HQL, et. Al Higher the risk for errors Java

ORM JPA Speedment- a Java Development Tool Connector JDBC Application Object Oriented functional programming  Fast to develop  Fast Execution  Easy to test  Secure  Make use of all Java 8 features Domain Model reflects your existing datasource Automatic generation of Code (”error free”) Java Speedment Java 8 Application

Speedment- a Java Development Tool

Code Examples • Embrace Java 8 paradigms with stream(), filter(), sort(), limit() etc. • Forget about SQL, JQL et. al

Database Table “hare” mysql> explain hare; +-------+-------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | name | varchar(45) | NO | | NULL | | | color | varchar(45) | NO | | NULL | | | age | int(11) | NO | | NU | | +-------+-------------+------+-----+---------+----------------+ 4 rows in set (0.01 sec)

Code Generation public interface Hare extends Entity<Hare> { public final static ReferenceComparableField<Hare, Integer> ID = … public final static ReferenceComparableStringField<Hare> NAME = … public final static ReferenceComparableStringField<Hare> COLOR = … public final static ReferenceComparableField<Hare, Integer> AGE = … Integer getId(); String getName(); String getColor(); Integer getAge(); Hare setId(Integer id); Hare setName(String name); Hare setColor(String color); Hare setAge(Integer age); /** Graph-like traversal methods eliminating JOINs */ Stream<Carrot> findCarrotsByOwner(); Stream<Carrot> findCarrotsByRival(); Stream<Carrot> findCarrots(); }

Querying using Java 8 Streams List<Hare> oldHares = hares.stream() .filter(Hare.AGE.greaterThan(4)) .collect(toList()); SELECT * from `hare` WHERE `age` > 4;

Stream Fundamentals Detailed description Slide 35-45

Depends on Terminating Operation Optional<Hare> oldHare = hares.stream() .filter(Hare.AGE.greaterThan(4)) .findAny(); SELECT * from `hare` WHERE `age` > 4 LIMIT 1;

Depends on Terminating Operation long noOldHares = hares.stream() .filter(Hare.AGE.greaterThan(4)) .count(); SELECT count(*) from `hare` WHERE `age` > 4;

Entities are linked Optional<Carrot> carrot = hares.stream() .filter(Hare.NAME.equal(”Spire”)) // Carrot is a foreign key. .flatmap(Hare::findCarrots) .findAny();

Multi-threading hares.stream() .parallel() .forEach(doExpensiveOperation());

Persistence Hare harry = hares.newInstance() .setName("Harry") .setColor("Gray") .setAge(3) .persist(); // OR JPA-style entityManager.persist(harry);

Delete Using Streams users.stream() .filter(User.ID.greaterThan(4)) .forEach(User::delete); DELETE FROM `user` WHERE `id` > 4;

Initialization Speedment speedment = new HareApplication().build(); Manager<Hare> hares = speedment.managerOf(Hare.class);

Speedment Insane Mode •Extreme Performance, real time big Data •Convert large databases into In-JVM-memory Java Objects •Scales Linearly -More CPUs, More Insane

In Memory Initialization Speedment speedment = new HareApplication() .with(OnHeapReadOnlyCacheComponent.class) .build();

How does it Work? 23 1 In-JVM-memory cache 2 Java Objects in Graph View 3 Separates Reads and Writes Reflection in real time Application JVM Read Write

Why will in-JVM-Memory give Insane Speed? Database In JVM Memory Compare latencies using the Speed of Light

Why will in-JVM Memory give Insane Speed? • Keep everything in-JVM • No need to check if an object is in cache • We can organize objects knowing that we have them all • We do not even have to look at individual objects -> O(1) • Everything is Java Size Time O(1) O(n)

Speedment – OSS and Enterprise

Cloud Deployment RDBMS Speedment Speedment Speedment Speedment Speedment • Spin up nodes as you grow • Partition data over nodes

How can I fit my full Database in JVM Memory? ~24 GB >1 TB >4 TB ∞ On-Heap Off-Heap SSD SQL Database Size (After Compression)

Scale out with our partner Hazelcast • Set up a cluster in the Cloud • Scale out your existing database • Set up Terabytes of RAM

Properties: • The API will remain the same regardless of selected Storage Engine • Massively concurrent and lock free • Scale with CPU performance and number of cores • Eventually consistent • Transaction safe, if you want

Configuration • Groovy script like Gradle • Config file is placed at the same location as the POM dbms { schema { table { name = ”hare"; column { name = "name"; } } } }

Ease of use: • Single Maven dependency in POM • Maven targets • Works with Gradle • GUI to automatically derive the groovy config file from existing databases

Properties: • The database ”owns” the data • Data is reflected into Materialized Object Views (MOVs) • MOVs and application resides in the same JVM • Most operations are O(1) • persons.byName(“Bob") -> Map<PK,Person> with 1000 persons in 100 ns (30 m) • TCP/IP RTT 100 us -> +1000 "finds" per TCP RT. • In-memory DBs with zero query latency are snails by definition

http://localhost:4567/trades/1001/200? from=0&to=1441147840000&callback=Ext.data.JsonP.callback1 Ext.data.JsonP.callback1([ {'start':1420131600000,'stop':1420236681000,'average':100876,'open':100000,'close':102209,'high':104060,'low':98714}, {'start':1420236682000,'stop':1420243199000,'average':102339,'open':102224,'close':103348,'high':103366,'low':101603}, {'start':1420477200000,'stop':1420502399000,'average':102272,'open':103353,'close':103316,'high':103608,'low':100819}, {'start':1420563600000,'stop':1420657005000,'average':101682,'open':103315,'close':102292,'high':103755,'low':99241}, … (200 rows in total) ... {'start':1441147839000,'stop':1441147839000,'average':142165,'open':142165,'close':142165,'high':142165,'low':142165} ]); Objectives:

Stream Fundamentals Per-Åke Minborg minborg@speedment.com

Setup 36 DB App Pwd ? In-JVM Cache

Stream<Trade>stream= trades.stream(); 37 DB Data Source Trade5 Trade4 Trade3 Trade2 Trade1

trades.stream().parallel(); 38 TradeN Trade4 Trade3 Trade2 Trade1 TradeX Thread 1 Thread 2 Thread N DB Data Source

trades.stream().filter(Predicate<Trade> predicate); 39 Trade2 Trade1 predicate. test(trade)Trade5 Trade4 Trade3 false true

trades.stream().map(Function<Trade,String>mapper); 40 Json2 Json1Trade5 Trade4 mapper. apply(trade) e.g.Trade::toJson

trades.stream().collect(Collectorcollector); 41 TradeN Trade4 Trade3 Trade2 Trade1 TradeX apply(trade) apply(trade) apply(trade)

Complete Stream 42 DB Data Source

Finding first and last date/time in an interval 43 from to selected

trades.stream().parallel() .filter(DATE.between(from, to)) .filter(STOCK.equal(stock)) .mapToLong(Trade::getDate) .summaryStatistics(); 44 2 3 4 2 3 4 2 3 4 5 1 2 3 4 5 DB Data Source

45 DB Data Source 1 4 4 4 5 2 3 trades.stream().parallel() .filter(DATE.between(from, to)) .filter(STOCK.equal(stock)) .mapToLong(Trade::getDate) .summaryStatistics(); 1 2 3 4 5

Generating a REST API Introduction to Speedment and Sencha Emil Forslund emil@speedment.com

Outline ●The Speedment + Sencha Software Stack ●Using the Pivotgrid Plugin for Speedment

Overview of the Software Stack DB Javascript Framework Server Framework DB Abstraction Layer

Database Layer ●Holds all the data collected from various sources ●High Level of Normalization Sale Product Salesperson Customer Country Region Office Cost Price Count

Database Abstraction Layer ●Presents an Object-Oriented view of the database ●Write expressive and powerful queries ●Optimize reading by using in-memory caching ●Better testability and higher productivity speedment.managerOf(Sale.class) .stream() .filter(Sale.PRODUCT.equal(105)) .collect(toList());

Server Framework ●A lightweight webserver ●Separates client/server-responsabilitys ●Execute selected business logic as response to REST commands ●Low Level of Abstraction http://localhost:8123/salesinfo/ ?c=callback &filter=[{...}]

Javascript Framework ●Presents data in user-friendly components ●User can navigate the data in the browser ●Automatically requests additional data from the server

Introduction to the Pivot Grid for Speedment ●Generate a full-fledged server application from your database ●Select exactly which data to expose in the API ●Define aliases for columns and tables ●Map between different data types ●Create virtual columns calculated in real-time

Step 1 – Create a New Project

Step 2 – Find Pivotgrid Archetype

Example Application: Salesinfo

DZone Java 8 Block Buster: Query Databases Using Streams

More Related Content

What's hot

Similar to DZone Java 8 Block Buster: Query Databases Using Streams

More from Speedment, Inc.

Recently uploaded

DZone Java 8 Block Buster: Query Databases Using Streams