Download as PDF, PPTX
Uploaded byRomeo Kienzler
Cognitive IoT using DeepLearning on data parallel frameworks like Spark & Flink
This document provides an overview of cognitive IoT anomaly detection using deep learning on sensor data. It discusses: - Deep learning and neural networks, including convolutional, recurrent and LSTM networks - DeepLearning4J for enterprise machine learning on sensor data - Examples of cognitive IoT services including anomaly detection for predictive maintenance using neural networks trained on historical IoT data - Parallelization of deep learning across CPUs, GPUs, and distributed systems like Apache Spark
Cognitive IoT using DeepLearning on data parallel frameworks like Spark & Flink
- 1. Unless stated otherwiseall images are taken from wikipedia.org or openclipart.org Cognitive IoT Anomaly Detector with DeepLearning4J on IoT Sensor Data @romeokienzler DevDays, Lithuania, 17th May 2017
- 2. • 6000+ clients •$200 million investment • Partners including • Avnet, BNP Paribas, EEBus, Capgemini, Tech Mahindra, Vodafone, BMW, Visa, Bosch, Indiegogo, French national railway SNCF,Arrow Electronics, Intel, Cisco
- 3. Why IoT (now)? • 15 Billion connected devices in 2015 • 40 Billion connected devices in 2020 • World population 7.4 Billion in 2016
- 18. Machine Learning on historicdata Source: deeplearning4j.org
- 19.
- 20. online vs. historic •Pros • low storage costs • real-time model update • Cons • algorithm support • software support • no algorithmic improvement • compute power to be inline with data rate • Pros • all algorithms • abundance of software • model re-scoring / re- parameterisation (algorithmic improvement) • batch processing • Cons • high storage costs • batch model update
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 39. Learning of afunction A neural network can basically learn any mathematical function
- 40.
- 41.
- 42. LSTM “vanishing gradient problem”== influence of past inputs decay quickly over time
- 43.
- 47.
- 48. •Outperformed traditional methods,such as •cumulative sum (CUSUM) •exponentially weighted moving average (EWMA) •Hidden Markov Models (HMM) •Learned what “Normal” is •Raised error if time series pattern haven't been seen before
- 51. Learning of analgorithm A LSTM network is Turing complete 1 1: http://binds.cs.umass.edu/papers/1995_Siegelmann_Science.pdf
- 52. Learning of analgorithm V2 The Differentiable Neural Computer
- 53. Problems • Neural Networksare computationally very complex •especially during training •but also during scoring CPU (2009) GPU (2016) IBM TrueNorth (2017)
- 54. IBM TrueNorth •Scalable •Parallel •Distributed •Fault Tolerant •NoClock ! :) •IBM Cluster •4.096 chips •4 billion neurons •1 trillion synapses •Human Brain •100 billion neurons •100 trillion synapses •1.000.000 neurons •250.000.000 synapses
- 57. Watson IoT Watson Cognitive Services Analytics Model +API Driver Behaviour Personality Insights Trainable Model + API IoT for Insurance Visual Recognition Customizable Model + API Watson Machine Learning Data Science Platform as a Service Data Science Experience
- 58.
- 59. Components • DeepLearning4J Enterprise GradeDeepLearning Library • DataVec CSV/Audio/Video/Image/… => Vector • ND4J / ND4S (NumPy for the JVM)
- 60. ND4J • Tensor support(Linear Buffer + Stride) • Multiple implementations, one interface • vectorized c++ code (JavaCPP), off-heap data storage, BLAS (OpenBLAS, Intel MKL, cuBLAS) • GPU (CUDA 7.5)
- 61.
- 62. DL4J parallelisation • TensorFlowon ApacheSpark => • Scoring • Multi-model hyper-parameter tuning • Parallel training since V r0.8 • DeepLearning4J => • Scoring, Multi-model hyper-parameter tuning • Parallel training “Jeff Dean style parameter averaging”
- 63. “Code local vsspark” vs.
- 64. Demo IoT / Industry/ Predictive Maintenance Use Case
- 68.