Low Latency Mobile Messaging using MQTT

1 Using MQ Telemetry Transport (MQTT) for Mobile Applications Low Latency Messaging for Mobile Apps – Or When HTTP and Push Notifications Are Simply Not Enough Henrik Sjöstrand CTO Mobility Nordic, WebSphere Group IBM Software Group, Sverige henrik@se.ibm.com

2 Agenda Impact of Mobile on the traditional Web MQ Telemetry Transport (MQTT) overview Protocol deep dive How do you get started?

3 Four major waves of technology – Mobile is the next transformation Back-Office Computing Client-Server PC - 1981 World Wide Web and eBusiness Confluence of Mobile, Social, Cloud, Big Data / Analytics 90’s80’s60’s We are here

6 Mobile is Different from Desktop Desktop Sit back and read Document-oriented Large complex apps Context-neutral Task-driven Mains powered Predictable network response Mobile Read while moving Message-oriented Purpose-built mini-apps Context-aware Notification-driven Battery-powered Unpredictable network response

7 Mobile is Driving Changes to the Traditional Web –  No built in reliability •  Loses data on brittle networks –  Polling oriented •  Unnecessary additional communications •  Wastes battery life –  Chatty •  Each message involves lots of back and forth –  One directional invocations •  Client always initiates the communications. HTTP –  Applications need to be online –  Browser plugins for additional application function (e.g., Flash, Silverlight) HTML4 HTML5 WebSockets? –  Offline execution –  No plugins required. –  Bi-directional data exchange •  Useful for push and streaming data (i.e. stock-ticker, media) –  But missing key function •  A low level TCP socket; not a full blown messaging protocol •  No assured delivery •  No built-in publish / subscribe •  Requires coding of message handling on top of it.

8 Notification Technologies for Mobile Mobile Vendors SMS §  Ubiquitous (supported by every phone/telco) §  Expensive, telcos charge per message §  Limited text-based notification §  In the clear (traffic read by Telcos) §  Not interoperable; different per Mobile OS –  APNS (Apple), GCM (Google), BBM (Blackberry) §  Requires a data plan §  Limited text-based notification §  Not suitable for confidential information (flows through Google, Apple, Blackberry) §  Transmission is one-way, not interactive §  No assured delivery

10 Facebook Messenger “One of the problems we experienced was long latency when sending a message. The method we were using to send was reliable but slow, and there were limitations on how much we could improve it. With just a few weeks until launch, we ended up building a new mechanism that maintains a persistent connection to our servers. To do this without killing battery life, we used a protocol called MQTT that we had experimented with in Beluga. MQTT is specifically designed for applications like sending telemetry data to and from space probes, so it is designed to use bandwidth and batteries sparingly. By maintaining an MQTT connection and routing messages through our chat pipeline, we were able to often achieve phone-to-phone delivery in the hundreds of milliseconds, rather than multiple seconds.” http://www.youtube.com/watch?v=aJo5jG0eKtI&hd=1 -Lucy Zhang, Facebook Engineer 10/19/11 www.facebook.com/lucyz (MQTT used by their 350M mobile users, 475 mobile operators)

14 MQTT: Key Features Reliable Open Simple Lean •  Open published spec designed for the world of “devices” •  Invented by IBM and Eurotech •  MQTT client code (C and Java) donated to the Eclipse "Paho" M2M project •  Simple / minimal pub/sub messaging semantics •  Asynchronous (“push”) delivery •  Simple set of verbs -- connect, publish, subscribe and disconnect. •  Minimized on-the-wire format •  Smallest possible packet size is 2 bytes •  No application message headers •  Reduced complexity/footprint •  Clients: C=30Kb; Java=100Kb •  Three qualities of service: •  0 – at most once delivery •  1 – assured delivery but may be duplicated •  2 – once and once only delivery •  In-built constructs to support loss of contact between client and server. •  “Last will and testament” to publish a message if the client goes offline. •  Stateful “roll-forward” semantics and “durable” subscriptions.

15 MQTT: Key Features §  240,000 concurrent clients tested with <5% CPU on a single IBM WebSphere MQ queue manager §  By comparison: –  Apache Web Servers max out at 25,000 connections §  Direct connection between your enterprise and devices §  Network: TLS/SSL §  Authentication: JAAS §  Authorization: OAM Secure Scalable

16 MQTT uses (much) less bandwidth than HTTP Scenario HTTP MQTT 1. Get a single piece of data from the server 302 bytes 69 bytes (~4 times) 2. Put a single piece of data to the server 320 bytes 47 bytes (~7 times) 3. Get 100 pieces of data from the server 12600 bytes 2445 bytes (~5 times) 4. Put 100 pieces of data to the server 14100 bytes 2126 bytes (~7 times) IBM Hursley Lab European automobile manufacturer Vehicle Telematics Mobile Network Estimated Data Costs/Vehicle/Year* HTTP MQTT 220€/vehicle /year 23€/vehicle /year *Comparison based on 100 messages/day, 200Bytes/Msg payload, 1-2€ /100MB TCP transfer costs.

17 MQTT Low Power Usage §  Restriction on some devices, such as cell phones may simply be available power, how does MQTT use power? –  HTC Desire Android mobile phone §  Protocol allows tuning to suit devices

19 MQTT Protocol Deep Dive - Headers §  MQTT protocol flows: –  Fixed header (2 bytes) –  Variable header (optional, length varies) –  Message payload (optional, length encoded, up to 256MB) §  Fixed header indicates the API call, the length of the payload and Quality of Service §  Variable header contents depends on API call defined in the fixed header –  Message ID, Topic name, client identifier and so on.

21 Qualities of Service §  QoS 0: At most once delivery (non-persistent) §  – No retry semantics are defined in the protocol. §  – The message arrives either once or not at all. §  QoS 1: At least once delivery (persistent, dups possible) §  – Client sends message with Message ID in the message header §  – Server acknowledges with a PUBACK control message §  – Message resent with a DUP bit set if the PUBACK message is not seen §  QoS 2: Exactly once delivery (persistent) §  – Uses additional flows to ensure that message is not duplicated §  – Server acknowledges with a PUBREC control message §  – Client releases message with a PUBREL control message §  – Server acknowledges completion with a PUBCOMP control message

25 MQTT Topics §  All subscriptions are to a topic space §  All messages are published to an individual topic §  Topic names are hierarchical –  Levels separated by “/” –  Single-level only wildcards “+” can appear anywhere in the topic string –  Multi-level (whole subtree) wildcards “#” must appear at the end of the string •  Wildcards must be next to a separator •  Can't use wildcards when publishing §  MQTT topics can be 64KB long Message Broker MQTT Client MQTT Client Flight Times/LHR/Air Freedom/Arrivals/Flight 123 Flight Times/LHR/+/Arrivals/+ ”publish” ”subscribe”

26 MQTT Keep Alive §  Protocol includes support for client and server to detect failed connections –  At connection time, a keep alive can be specified §  If client does not send a PINGREQ request to the server, the server assumes the connection has failed. §  If client does not receive a PINGRESP after the keep alive time has passed after sending the request, the connection is failed. §  Maximum keep alive interval of 18 hours. –  Can specify a value of 0 to disable keep alive ?

27 MQTT Last Will & Testament §  During connection, a Will message and topic can be specified –  Abnormal disconnections will cause WMQ to publish the message –  Clean disconnects will not cause the message to publish §  Can set the message as retained –  Message is published to a subscriber when registering §  Useful to report the connection status of the client –  Will message is a retained “down” –  Upon connecting, client publishes a retained “up” message.

28 Agenda Impact of Mobile on the traditional Web MQ Telemetry Transport (MQTT) overview Protocol deep dive How do YOU get started?

29 Servers/Brokers § IBM WebSphere MQ (incl Telemetry) § IBM WebSphere Message Broker § IBM Lotus Expeditor micro broker § Really Small Message Broker § Mosquitto § Eclipse Paho § Eurotech Everyware Device Cloud (EDC) § Pachube § MQTT.js § eMQTT § m2m.io Clients § Device-specific - Arduino client for MQTT - mbed client for MQTT - Nanode MQTT - Netduino MQTT § C § C++ § Delphi § Erlang § Java § JavaScript / node.js § LotusScript § Lua § .NET § Perl § PHP § Python § REXX § Ruby

31 ■  Native Android •  Use the MQTT Java client and samples •  This client provides synchronous and asynchronous APIs for MQTT •  Available as open source from Eclipse Paho, or as supported code from IBM •  IBM provides a sample Android service, that lets the MQTT client continue running even when the app is in the background ■  Native iOS •  Use the MQTT C Client and samples. This can be called from Objective C •  This client provides synchronous and asynchronous APIs for MQTT •  Available as open source from Eclipse Paho, or as supported code from IBM ■  JavaScript •  For applications running in Web Browsers (mobile or static) •  For cross-platform hybrid apps (Android, iOS, Windows Phone or BB10) •  Clients provide asynchronous APIs for MQTT MQTT clients for Native and JavaScript Mobile Apps

32 ■  WebSocket implementation – the Messaging Client for JavaScript •  Implementation is 100% JavaScript (single JavaScript file to include in your app) •  Ideal for Browser-based apps (downloads with the app itself) •  Supported on all modern Web Browsers •  Runs in iOS 6.0 and Android 4.2.3, Blackberry 10, Windows Phone 8 •  Requires MQ 7.5.0.1 or MessageSight server implementation ■  Cordova implementation – the Cordova plugin for Android •  Uses Cordova technology to bridge between Javascript and the native Paho client •  Runs on all versions of Android from 2.2 onwards •  Allows app to continue running in the background •  Runs against MessageSight and all versions of MQ from 7.1 upwards ■  Both clients present the same API to app developer •  Single line change in the HTML file to select which client to load JavaScript options

33 MQTT Broker Commercial alternative 1: IBM WebSphere MQ includes Telemetry component §  New WebSphere MQ Telemetry component – also known as MQXR ('eXtended Reach') §  Included in WMQ since v7.1 §  Fully integrated / interoperable with WMQ –  MQTT messages translated to standard WMQ messages –  Administration included as part of WebSphere MQ Explorer §  Telemetry channels enable MQTT connections to the queue manager –  Supports MQTTv3 protocol (most common in use) §  Highly scalable –  100,000+ clients §  Rich Security –  SSL channels –  JAAS authentication §  Ships with reference Java (for MIDP upwards) and C clients –  Small footprint clients –  other APIs and implementations of MQTT available via 3rd parties http://www.ibm.com/software/integration/wmq/

34 § A secure, easy to deploy messaging server that is optimized to address the massive scale requirements of the machine to machine (M2M) and mobile use cases § A million connections, and millions of messages per second § Designed to sit at the edge of the enterprise, it can extend your existing messaging infrastructure or be used standalone. § Connectivity to existing WebSphere MQ and to messaging clients using MQTT and JMS MQTT Broker Commercial alternative 2: IBM MessageSight

3636 © 2013 IBM Corporation Client Object – represents the connection with the server ■  Properties • Hostname, Port, ClientId ■  Events – onMessageArrived, onMessageDelivered, onConnectionLost ■  Methods • Connect, Disconnect, Send, Subscribe, Unsubscribe Message Object – represents data sent or received ■  DestinationName ■  Quality of Service (QoS) ■  Retained flag ■  Duplicate flag ■  Payload (string or binary) JavaScript Messaging API

3737 © 2013 IBM Corporation function connect(form) { try { client = new Messaging.Client( form.host.value, Number(form.port.value), form.clientId.value); } catch (exception) { alert("Exception:"+exception); } client.onConnect = onConnect; client.onMessageArrived = onMessageArrived; client.onConnectionLost = connectionLostCallback; client.connect(); } Connect to the server Create client Set callbacks Connecting to the Server

3838 © 2013 IBM Corporation function doSend(form) { message = new Messaging.Message(form.textMessage.value); message.destinationName = form.topicName.value; client.send(message); } Send the message Create a Message object Set the Topic Sending a Message

3939 © 2013 IBM Corporation function subscribe(form) { client.subscribe(form.subscribeTopicName.va lue); } function onMessageArrived(message) { var form = document.getElementById("output"); form.receivedMessage.value = message.payloadString; } Subscribe to a topic Subscribing and Receiving Display the message’s payload

4040 © 2013 IBM Corporation var subscribeOptions = {qos:2, invocationContext:{text:"Subscription 1"}, onSuccess:onSuccess, onFailure:onFailure}; client.subscribe("/World", subscribeOptions); }; function onSuccess(resp) { console.log("OK:"+resp.invocationContext.text); }; function onFailure(resp) { console.log("Failed:"+resp.invocationContex t+" "+resp.errorCode +" "+ +resp.errorReason); }; Success and Failure Success / Failure are signalled by callbacks registered when the API call is made. You can associate an InvocationContext with a callback

4141 © 2013 IBM Corporation Function connect(form) { … client.connect({onSuccess:onConnect}); // client won’t yet be connected ! } function onConnect() { console.log("it’s now connected"); client.subscribe("/World"); message = new Messaging.Message("Hello"); message.destinationName = "/World"; client.send(message); }; Asynchronous programming Warning: API calls like connect and subscribe are Asynchronous. You can’t assume that they have completed until you have been called back. In single threaded environments (like phones) this won’t happen till after you have relinquished control of your thread

47 Links §  MQTT.org - documentation and software http://mqtt.org §  Eclipse paho project (IBM’s MQTT reference implementation) http://www.eclipse.org/paho/ §  IBM WebSphere MQ http://www.ibm.com/software/integration/wmq/ §  IBM Worklight http://www.ibm.com/software/mobile-solutions/worklight/

48 © Copyright IBM Corporation 2012. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. Henrik Sjöstrand CTO Mobility Nordic, WebSphere IBM Software Group, Sverige henrik@se.ibm.com

Low Latency Mobile Messaging using MQTT

More Related Content

What's hot

Similar to Low Latency Mobile Messaging using MQTT

Recently uploaded

Low Latency Mobile Messaging using MQTT