Abhishek Lal | Senior Program Manager, Azure Service BusMiranda Luna | Product Manager, Windows Azure Mobile

Exploring M2M Information Exchange Patterns using Azure Service Bus

Agenda

M2M Information Exchange Patterns using Service Bus01 | Introduction to M2M 04 | Notifications

02 | Key Concepts 05 | Command/Control/Inquiry

03 | Telemetry 06 | Resources

Join the MVA Community!▪ Microsoft Virtual Academy—Free online training!

▪ Ask questions in the Born to Learn MVA Forum!‒ Visit http://aka.ms/MVAForum

▪ Earn while you learn! ‒ 50 MVA Points for this event!‒ Visit http://aka.ms/MVA-Voucher ‒ Code: BldgBlks3

Click to edit Master subtitle style

01 | Introduction to M2M

What are devices?

• Special purpose devices (not general purpose computers)

• Small footprint – sensors, controllers, wearable

• Diverse processor architectures

• Non-standard Operating Systems

• Specialized development environments

• Some hobbyist platforms – – .NET Gadgeteer – Arduino

Connectivity considerations

Styles Directionality Types Metering

Protocols Approach Location Power

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

Patterns of communication

Telemetry

Information flowing from a device to other systems for conveying status of device and environment

Inquiries

Requests from devices looking to gather required information or asking to initiate activities

Commands

Commands from other systems to a device or a group of devices to perform specific activities

Notifications

Information flowing from other systems to a device (-group) for conveying status changes in the rest of the world

1:N 1:N

Click to edit Master subtitle style

02 | Key concepts

Windows Azure Service Bus 101• Queues– brokered messaging communication model– components of an application don’t communicate directly, but rather

exchange messages via an intermediary queue– One-to-one, asynchronous, follows FIFO

• Topics & Subscriptions– pub/sub messaging communication model– components of an application don’t communicate directly, but rather

exchange messages via an intermediary topic– one-to-many, can register multiple subscriptions to a topic, each

message sent to a topic is available for each subscription to handle independently

Queues

• Decouple sender and receiver

• One or many senders.

• One or many (competing) receivers.

Load leveling Receiver is never overloaded, process at its own pace.

Temporal decoupling Allows taking the receiver offline for servicing.

Load balancing Multiple receivers compete for messages.

Sender Receiver

SenderReceiver

Sender

Sender

Receiver

Topics & Subscriptions

• Senders send to a topic

• Receivers receive from subscription

• Each subscription has one copy of the message.

• Filters, Rules and Actions Message distributionDeliver the same message to more than one client. Subscribers can filter by interest.

Message partitioningReceiver get mutually exclusive slices of the message stream.

Sender

Receiver

Sender

Sender Receiver

F1

F2

Receiver

Filters

• Filters act on message properties.

• Up to 2000 rules per subscription.

• SQL ‘92 expressions over message properties.

• Filter action allows to add/remove/change message properties.

• Partition by filtering on mutually exclusive ranges.

Service Assisted Communication (aka.ms/iot5)

Connections are device-initiated and outbound

(like VPN)

NAT/FW Device (Router)

IPv4 NAT

Service GatewayClient

DNS+

Device Mapped via Mplx Protocol or Port

Port Mapping is automatic, outbound

(like VPN)

Device does not actively listen for unsolicited traffic

(unlike VPN)No inbound ports open,

attack surface is minimized

Public address, full and well defendable server

platform

Service Bus a Polyglot Messaging Service

“SBMP”high perf..NET only

AMQP 1.0high perf

high reach

App ( any language )

Community LibsAMQP 1.0

Any OS

App ( .NET )

SB .NET Lib“SBMP”Windows

App ( any language )

SB Wrappers

HTTPAny OS

HTTPhigh reachlower perf.

Binary Protocols Matter with Messaging

• HTTP• SBMP/AMQP

Client Client

Q S

T

N Q S

T

N

Socket Socket

HTTP 1 Entity per Socket 1 Pending Operation per Socket 60s operation timeout (NAT/Prx)

SBMP/AMQP Unlimited Multiplexed Entities and Unlimited Pending Ops per Socket No fixed operation timeout Session Support (coming in AMQP)

AMQP 1.0 Standardization

OASIS Standard since October 2012The culmination of several years effort by more than 20 companiesTechnology vendors: Axway Software, Huawei Technologies, IIT Software, INETCO Systems, Kaazing, Microsoft, Mitre Corporation, Primeton Technologies, Progress Software, Red Hat, SITA, Software AG, Solace Systems, VMware, WSO2, Zenika. User firms: Bank of America, Credit Suisse, Deutsche Boerse, Goldman Sachs, JPMorgan Chase

Next is standardization via ISO/IEC JTC1

Platforms, Protocols and Client libraries

WindowsSBMP(net.tcp,

proprietary)

Service Bus

Queues &

TopicsHTTP(S)

AMQP 1.0

Other platforms

https://github.com/windowsAzure/

/azure-sdk-for-python/

/azure-sdk-for-php/

/azure-sdk-for-node/

/azure-sdk-for-java/

/azure-sdk-for-ruby/

AMQP 1.0

HTTP(S)

Samples / Prototypes

Click to edit Master subtitle style

03 | Telemetry

Patterns of communication

Telemetry

Information flowing from a device to other systems for conveying status of device and environment

Inquiries

Requests from devices looking to gather required information or asking to initiate activities

Commands

Commands from other systems to a device or a group of devices to perform specific activities

Notifications

Information flowing from other systems to a device (-group) for conveying status changes in the rest of the world

1:N 1:N

Messages

• Brokered messaging properties are not SOAP headers

• Properties are key/value pairs that may very well carry payloads

• It’s not uncommon to have messages with empty message bodies

• Message bodies are useful for a single opaque payload not exposed to the broker (e.g. encrypted content)

Broker Message

Body

Properties

Key Value

Key Value

Key Value

Key Value

Body

Telemetry

• Scheduled or event-driven stream of device status information

• Physical or logical status or sensor readings.

• Records are usually tiny; timestamp plus numbers.

• Grand variety in frequencies. Once per week to 10 kHz for each of multiple metering points for factory machines.

Telemetry Types

•

•

•

•

•

•

DEMOUsing device to send telemetry messages

Click to edit Master subtitle style

03 | Notifications

Patterns of communication

Telemetry

Information flowing from a device to other systems for conveying status of device and environment

Inquiries

Requests from devices looking to gather required information or asking to initiate activities

Commands

Commands from other systems to a device or a group of devices to perform specific activities

Notifications

Information flowing from other systems to a device (-group) for conveying status changes in the rest of the world

1:N 1:N

Notification

• Scheduled or event-driven message to device

• Informs a device, remotely, about a relevant change or event– “Status of X has changed” – “You have reached your target”– “An update has been released”

• Remote: Control service, handheld device, etc.

• Latency requirements vary, from “real-time” to occasionally connected

• Messages are usually tiny, metadata / data both may be included

• Grand variety in frequencies. Once per year to few seconds

Broadcast Fan-Out

Fan-Out

Sub Sub SubSub Sub SubSub Sub Sub

Topic Topic Topic

Partition

Control System

Broadcast Fan-Out

Fan-Out

Sub Sub SubSub Sub SubSub Sub Sub

Topic Topic Topic

Partition

Control System

Device and Housing Unit

based filter for the device

subscriptionDeviceId = 12345 ORHousingUnit = 8821 ORBroadcast = true

Create Subscriptions with Rules (Filters)

TopicDescription mainTopic = namespaceManager.CreateTopic(“topicName");

namespaceManager.CreateSubscription(“topicName", “AuditSubscription"); namespaceManager.CreateSubscription(T“topicName", "FirstSubscription",

new SqlFilter("Address LIKE '%First%'"));

namespaceManager.CreateSubscription(T“topicName", “SecondSubscription", new SqlFilter("Address LIKE '%Second%'"));

BrokeredMessage myMessage = new BrokeredMessage(); myMessage.Properties.Add(“Address”, “First”);

or myMessage.Properties.Add(“Address”, “Second”);

or myMessage.Properties.Add(“Address”, “First,Second”);

DEMONotifications with Topics

Click to edit Master subtitle style

03 | Command/Control/Inquiry

Patterns of communication

Telemetry

Information flowing from a device to other systems for conveying status of device and environment

Inquiries

Requests from devices looking to gather required information or asking to initiate activities

Commands

Commands from other systems to a device or a group of devices to perform specific activities

Notifications

Information flowing from other systems to a device (-group) for conveying status changes in the rest of the world

1:N 1:N

Command/Control/Inquiry

• Tell a device or service, remotely, to execute a logical or physical activity– “Give me the status of X” – “Roll 2 feet forward”– “Track this object with the camera”– “Fetch firmware update”

• Remote: Control service, handheld device, etc.

• Latency requirements vary, but often “perceptibly imminent”

Request/Response channels

• Single ingress and egress endpoint from device perspective

• Can reuse Ingress queue across devices (fan-in)

• Message payload properties used to capture metadata on device, command, status

• Device is always addressable via Queue even if not connected

• Ingress Queue provides load balancing/leveling

QIQE

DEMORequest/response correlation

Click to edit Master subtitle style

03 | Resources

Summary and next steps..

• Covered key concepts and Service Bus basics

• Identified patterns of communication and implementation details

• Achieved unidirectional and bidirectional communication

• Next Steps: Scale to millions of devices and messages

M2M at //build

• Internet of Things (IoT) with Azure Service Bus– Wednesday 4/2 2:30 – 3:30 PM PST– Todd Holmquiest-Sutherland, Paolo Salvatori– @Skraelinger, @babosbird

• Powerful mobile apps with Mobile Services and ASP.NET Web API– Friday 4/4 2:00 – 3:00 PST– Abhishek Lal– @AbhishekRLal

• All videos will be available on Channel9

Windows Azure

Device to Cloud ResourcesService Bus

Overview

Documentation

Tutorials

AMQP 1.0 supportOverview

Developer Guide

.NET Tutorial

Java Tutorial

Clemens Vasters Device to Cloud Series on Channel 9Part 1 Prototyping Platforms Part 2 Pattern Overview and Commands with HTTPPart 3 Safer Commands via a Cloud GatewayPart 4 Intermediated, Service-Assisted Connectivity Part 5 Tunnel in Tunnel in TunnelPart 6 Why End-to-End Security MattersPart 7 The Pi in the CarPart 8 OBDII to AMQP to Cloud

©2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.