Download as PDF, PPTX
![Device Manager: Devices : Identity Device identity is based on an asymmetric key-pair of two supported formats: ● RSA 256 public key wrapped in a X.509v3 certificate ● Elliptic curve (ECDSA) algorithm using P-256 and SHA-256 [more efficient, better suited for small devices] Credentials may optionally have an expiration timestamp A device can have up to 3 credentials associated with it at a time, allowing for rotation The service should never need the private key The sequence shown here is only one way to handle device provisioning](https://image.slidesharecdn.com/googlesinfrastructureandspecificiotservices-171017173237/75/Google-s-Infrastructure-and-Specific-IoT-Services-23-2048.jpg)
Uploaded byIntel® Software
Google's Infrastructure and Specific IoT Services
This document discusses Google Cloud Platform's Internet of Things (IoT) solutions. It describes IoT Core, which handles device management and communication, including the Device Manager for registering devices and MQTT Broker for bidirectional messaging. It explains how IoT Core collects analog sensor data from devices and transforms it into useful business insights and intelligence through data processing and analytics services like Cloud Dataflow, BigQuery, and Cloud ML.
Google's Infrastructure and Specific IoT Services
- 1. Preston Holmes Head ofIoT Solutions IoT at Google Cloud
- 2. Google Cloud Platform2 Organize the world’s information and make it universally accessible and useful. Google’s Mission 2 “
- 3. Google Cloud Platform3 We’re generating more data than ever before The number of connected “things” in use in 2017, up 31% from 2016* 8.4B Confidential & ProprietaryGoogle Cloud Platform 3
- 4. Phone"IoT"Phone IoT is aperiod of transformation
- 5. IoT is aperiod of transformation Wearables Connected Not Connected Watches Home Appliances Phones Cars Existing Business Owned Equipment
- 6. BuildingsTransportation Healthcare Retail Manufacturing &Industrial CitiesHome Information is Everywhere But it’s not DATA Yet
- 7. How do youcollect and process this analog information, to transform into useful business Intelligence? InsightInformation
- 8. How do youcollect and process this analog information, to transform into useful business Intelligence? DigitalAnalog InsightInformation
- 9. How do youcollect and process this analog information, to transform into useful business Intelligence? DigitalAnalog InsightInformation Internet of Things
- 10. Confidential & ProprietaryGoogleCloud Platform 10 A datacenter is not a collection of computers, a datacenter is a computer. Confidential & ProprietaryGoogle Cloud Platform 10
- 11.
- 13. Confidential & ProprietaryGoogleCloud Platform 13 Google Jupiter: On the left is a Jupiter superblock. It is one part of a fabric which interconnect up to 100,000 servers at 10 Gbit/s each—more than 1 Petabit/sec of total bisection bandwidth for transfer of information between physical and virtual machines It’s enough to transfer the entire scanned contents of the library of congress in 1/10 of a second.
- 14. Google Global Cache (GGC)edge nodes Points of presence (>100) Network fiber FASTER (US, JP, TW) 2016 Unity (US, JP) 2010 SJC (JP, HK, SG) 2013 Monet (US, BR) 2017 Google Network More than a collection of data centers
- 15. Google Cloud Platform15 2004 2006 2008 2010 2012 2014 2015 Our research and legacy in data management and analytics stack run deep Open Source
- 16. Google Cloud Platform16 GFS TensorFlowBigTable MapReduce Dremel Flume/Java Spanner Millwheel PubSub 2002 2004 2006 2008 2010 2012 Our research and legacy in data management and analytics stack run deep Google Papers + Products BigQuery Pub/Sub Dataflow Bigtable ML Spanner 2014 2015 2016
- 17. Google Big DataPattern Cloud Logs Google App Engine Google Analytics Premium Cloud Pub/Sub BigQuery Storage (tables) Cloud Bigtable (NoSQL) Cloud Storage (files) Cloud Dataflow BigQuery Analytics (SQL) Capture Store Analyze Batch Process Stream Cloud Monitoring Real-time analytics Cloud Dataflow Cloud ML Real-time dashboard Real-time alerts Use Data Scientists Analysts Smart apps Catalog & Data Lifecycle Automation Cloud Datalab Cloud Dataproc Data Studio
- 18. IoT Core handlesdevice management and bi-directional device communication as part of an over all GCP IoT Solution
- 19. What Services arein IoT-Core ? Device Manager ● Maintains registries of devices as named logical resources ● Protects these entries with IAM permissions ○ eg who can delete a device ● Associates device credentials with these resources ● Acts as the identity provider (IdP) for the MQTT broker ● Provides storage and delivery API for device configurations ● Maintains some operational state metadata for the device: ○ enabled/disabled ○ connectivity and error status MQTT Broker ● Provides a stateful socket connection to devices for bidirectional communication ● Brokers device->cloud telemetry data onto Cloud PubSub for use in downstream GCP products or customer applications ● Delivers configuration updates via a Device Manager API ● Is exposed through a global DNS endpoint over multiple ports
- 20. Google Cloud Platform20 Simple UI for monitoring and management Cloud Platform
- 21. What is MQTT MQTTis a machine-to-machine (M2M)/"Internet of Things" connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. ● Originally developed at IBM in 1999 and designed for constrained devices ● Is now a ratified standard ● Is a binary protocol making efficient use of over-the-wire bandwidth ● Is simple to implement and so has many small memory footprint libraries available ● Has become a common and de-facto standard used in many IoT projects
- 22. Project Registry Device Device Credential Device Config DeviceManager Resource Model The device manager organizes cloud resources to handle device management.
- 23. Device Manager: Devices: Identity Device identity is based on an asymmetric key-pair of two supported formats: ● RSA 256 public key wrapped in a X.509v3 certificate ● Elliptic curve (ECDSA) algorithm using P-256 and SHA-256 [more efficient, better suited for small devices] Credentials may optionally have an expiration timestamp A device can have up to 3 credentials associated with it at a time, allowing for rotation The service should never need the private key The sequence shown here is only one way to handle device provisioning
- 24. MQTT Broker: authentication Adevice authenticates with two pieces of information: 1. the MQTT client ID (a feature of the MQTT protocol) which must be in the form of device name: projects/{project-id}/locations/{cloud-region}/registries/{registry-id}/devi ces/{device-id} 2. An MQTT password in the form of a JWT token signed by the device's private key ● The "username" field in MQTT clients is ignored ● JWT token may have a max expiration of 1 hour ● Device's clock must be within 10 minutes of Google's time (use Google NTP)
- 25. Dataflow: organize torrentsof IoT data into actionable windows Input Output
- 26.
- 27.
- 28. Windowing divides datainto event-time-based finite chunks. Often required when doing aggregations over unbounded data. Where in event time? What Where When How Fixed Sliding 1 2 3 54 Sessions 2 431 Key 2 Key 1 Key 3 Time 1 2 3 4
- 29. When in processingtime? What Where When How • Triggers control when results are emitted. • Triggers are often relative to the watermark.
- 30. When: Triggering atthe Watermark What Where When How
- 31. 1.Classic Batch 2.Batch with Fixed Windows 3. Streaming 5. Streaming With Accumulations 4. Streaming with Speculative + Late Data Customizing What When Where How What Where When How
- 32. a unified modelfor batch and stream processing supporting multiple runtimes a great place to run Beam Apache Beam Google Cloud Dataflow The Dataflow Model & Cloud DataflowBeam
- 33. 33 one important technologywe use is neural networks OUTPUTINPUT
- 34. 34 neural net modelslearn from examples labeled photos “cat” “dog” “car” “apple” “flower” OUTPUT
- 35. 35 neural net modelslearn from examples Make tiny adjustments to model so output is closer to label for a given image labeled photos “cat” “dog” “car” “apple” “flower” OUTPUT
- 36. 36 after a modelis trained, you can test it ? unlabeled photo
- 37. 37 after a modelis trained, you can test it unlabeled photo “cat”
- 38. Google Cloud Platform38 Can I Hug That?
- 39. 39 Input Output “rice” “restaurants inSeoul” “hello!” “A close up of a small child holding a stuffed animal.” powerful functions that neural nets can learn 안녕하세요
- 40. 40 Rapidly accelerating useof deep learning at Google Number of directories containing model description files 2012 2013 2014 2015 1500 1000 500 0 Used across products:
- 41. Our new CloudTPU delivers up to 180 teraflops to train and run machine learning models.
- 42. Google Cloud Platform42 Get started by visiting - cloud.google.com/IoT-Core Confidential & ProprietaryGoogle Cloud Platform 42