UNIVERSITY OF JYVÄSKYLÄ

Lecture 5: Web of Devices & Sensors

TIES4371 Interface of ThingsSpring 2018

University of Jyväskylä Khriyenko Oleksiy

UNIVERSITY OF JYVÄSKYLÄ

Web of Devices & Sensors

225/04/2018 TIES4371 - Lecture 5

Relevant links:https://www.youtube.com/watch?v=WtRpFLx34BYhttps://www.youtube.com/watch?v=G4-CtKkrOmchttps://ocw.cs.pub.ro/courses/iot

UNIVERSITY OF JYVÄSKYLÄ

Web of Devices & Sensors

325/04/2018 TIES4371 - Lecture 5

Relevant links:https://www.youtube.com/watch?v=WtRpFLx34BY

…Protocol ComparisonCommunication:

Functionality:

UNIVERSITY OF JYVÄSKYLÄ

MQTT stands for Message Queuing Telemetry Transport. It is a publish/subscribe,extremely simple and lightweight messaging protocol, designed for constraineddevices and low-bandwidth, high-latency or unreliable networks. The design principlesare to minimize network bandwidth and device resource requirements whilst alsoattempting to ensure reliability and some degree of assurance of delivery. Theseprinciples also turn out to make the protocol ideal of the emerging “machine-to-machine” (M2M) or “Internet of Things” world of connected devices, and for mobileapplications where bandwidth and battery power are at a premium.Link: http://mqtt.org and https://en.wikipedia.org/wiki/MQTT

MQTT

4

� MQTT was invented by Dr Andy Stanford-Clark of IBM, and ArlenNipper of Arcom (now Eurotech), in 1999.

� MQTT has been widely implemented across a variety of industriessince 1999. A few of the more interesting examples are listed on theprojects page (http://mqtt.org/projects).

� Other FAQ (http://mqtt.org/faq)

MQTT-SN stands for “MQTT for Sensor Networks” which is aimed at embedded devices on non-TCP/IP networks, such as Zigbee, Z-Wave, etc. It is lighter and even more optimized for smallermemory and power consumption… (http://emqttd-docs.readthedocs.io/en/latest/mqtt-sn.html)

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

MQTT

5

Relevant links:https://www.hivemq.com/blog/how-to-get-started-with-mqtthttps://www.youtube.com/watch?v=TDFp9C1f2U0https://www.youtube.com/watch?v=Pb3FLznsdwIhttps://www.youtube.com/watch?v=EIxdz-2rhLshttps://www.youtube.com/watch?v=r6HEQVhgnP8

Topic is a simple string that can have morehierarchy levels, which are separated by a slashhouse/living-room/temperatureo Single level wildcard (+) allows arbitrary values for

one level hierarchy: house/+/temperature (includingliving-room, kitchen, etc.)

o Multilevel wildcard (#) allows to subscribe to allunderlying hierarchy levels: house/#

MQTT protocol uses a publish/subscribe architecture in contrast to HTTP with itsrequest/response paradigm. Publish/Subscribe is event-driven and enables messages to bepushed to clients.

MQTT broker is the central communicationpoint that is in charge of dispatching allmessages between the senders and therightful receivers. There are free selfhosted brokers, the most popular beingMosquitto and Mosca ,and commercialones like HiveMQ.Each client that publishes a message(command or data) to the broker, includesa topic into the message. Each client thatwants to receive messages subscribes to acertain topic and the broker delivers allmessages with the matching topic to theclient.

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Node-RED is a visual programming tool for wiring together hardware devices,APIs, storages and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodesin the palette that can be deployed to its runtime in a single-click.Link: https://nodered.org

Node-RED

6

Flow-based programming for the Internet of Things:� JavaScript functions can be created within the editor using a rich text editor.� A built-in library allows you to save useful functions, templates or flows for re-use.

Relevant links:https://www.youtube.com/watch?v=3AR432bguOYhttps://www.youtube.com/watch?v=j9H3t3l1En0

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

MQTT and Node-RED

725/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Node-RED and IBM Bluemix

8

Relevant links:https://www.youtube.com/watch?v=j9H3t3l1En0https://www.slideshare.net/MarkusVanKempen/simple-internet-of-things-iot-game-with-bluemix-and-nodered

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

IBM Bluemix is a cloud platform for rapidly building, running, andmanaging applications. The goal is:� to simplify the delivery of an application by providing hosting capabilities and services that are

ready for immediate use.

� helping to test and adopt a broad range of cloud services and capabilities from IBM, opensource communities, and third-party developers.

Link: https://www.ibm.com/cloud-computing/bluemix/

IBM Bluemix

9

Access to the services:� Sign up to the platform by creating your personal IBMid… https://console.bluemix.net/registration/

o 30-day trial is free.

o Get access to 2-8GB of runtime and container memory to run apps, unlimited IBMservices and APIs, and complimentary support.

� Login to the platform and go to the catalog of the offered services https://console.bluemix.net/catalog/

Extend you free access with additional 6 month trial promo codeobtained from Academic Initiative.https://ibm.onthehub.com/WebStore/OfferingDetails.aspx?o=bb3528b7-2b63-e611-9420-b8ca3a5db7a1/

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Gather Internet of Things sensor data…

� Collect sensor data from a mobile device

� Create a Node-RED application running on SDK for Node.js runtime

� Deploy a simple application from the IBM Bluemix

� Set up simple user interface for the application to display the sensorvalues

Bluemix and Node-RED

10

…gather Internet of Things sensor data

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Visualize sensor data from a mobile device using IBM Internet of Thingsfoundation quick start:

Bluemix and Node-RED

11

…gather Internet of Things sensor data

� Set up your mobile device to send its sensor data to the IBM Internet ofThings Foundation. Open a browser on your mobile device and go tohttp://phonesensor.mybluemix.net

� Remember the number presented after “ d:quickstart:phone-sensor ”� Navigate to https://quickstart.internetofthings.ibmcloud.com/#/ on your

computer and enter the number from the previous step

� Keep the browser with https://phonesensor.mybluemix.net open on yourphone and watch the sensor data flow in the IBM Internet of ThingsFoundation quick start window. Try moving your phone and selecting thedifferent phone sensors...

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Create Node-RED Application:

Bluemix and Node-RED

12

…gather Internet of Things sensor data

� Log in into your IBM Bluemix account (https://bluemix.net) and navigate to ‘CATALOG’ view and select Node-RED Starter from the boilerplate section…

� Choose a unique name for your application (since host will automatically be completed on Bluemix cloudbased on the name) and press ‘CREATE’. Your application will be staging, up and running in a short while.Press ‘OVERVIEW’ from menu on the left to see information about your application.

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Create Node-RED Application:

Bluemix and Node-RED

13

…gather Internet of Things sensor data

� When fully staged (your application is in status ‘Running’), click theApp URL for your application to launches the Node-RED main page.

� Go to your Node-RED flow editor to open the flow editor.

� Build apps using this graphical editor interface to wire together theblocks we need. Simply drag and drop the blocks from the left menuinto the workspace and connect them to create a new flow.

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Create Node-RED flow:

Bluemix and Node-RED

14

…gather Internet of Things sensor data

� Select the “ibmiot” node under the input section in the palette window on the left and dragthe node onto the workspace in the middle of the screen…

� Double click on the node and select ‘Quickstart’ under ‘Authentication’ and enter the phone-sensor number (obtained for your phone on one of the previous steps) in ‘Device Id’.

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Create Node-RED flow:

Bluemix and Node-RED

15

…gather Internet of Things sensor data

� Drag the “debug” node under the output section onto the workspace and connect it with“ibmiot” node by dragging the output and input dots together…

� Click ‘DEPLOY’ in the upper right corner.� Check the “debug” tab on the right of the screen to see the phone sensor data flowing.

Since the web application is only sending sensor data to Bluemix as long as the browser iskept open and the screen is on, make sure you have the phone sensor browser tab openon your phone.

� Move your phone up/down or sideways to see the sensor data changing in real time…

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Add extra functionality to the flow:

Bluemix and Node-RED

16

…gather Internet of Things sensor data

� Add a “function” node under function section in the palette window between the “ibmiot”and the “debug” nodes. In the function node, add the following code that will create theoutput “Phone rotated more than 45 degrees” in the “debug” tab if the phone is rotatedmore than 45 degrees (tiltFB is the tilt in the front-back direction of the phone).

� ‘DEPLOY’ the changes and try rotating your phone in the front-back direction. Try alsodifferent functions and outputs by editing the function node.

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Add a simple GUI to the app:

Bluemix and Node-RED

17

…gather Internet of Things sensor data

� Add some new UI components to the Node-RED palette. Click the button onthe top right corner and select ‘Manage palette’. Search for ‘node-red-contrib-ui’ and click ‘install’ button on the first node-red-contrib-ui entry in the list.

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

� Find the newly installed UI components at the bottom of the Node-RED palette scrolling itdown and add a new ‘Gauge’ node to the workspace.

� Double click the ‘Gauge’ node to edit it’s properties.

� Add another “function” node to the workspace and connect it to “ibmiot” and “gauge”nodes.

Add a simple GUI to the app:

Bluemix and Node-RED

18

…gather Internet of Things sensor data

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

� Open a new browser window and go to an URL where the newly created UI is visible - the same URL as theapplication’s Node-RED editor URL with replacement of ‘red/#’ to ‘ui’ at the end of the URL.

Access GUI of the app:

Bluemix and Node-RED

19

…gather Internet of Things sensor data

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Camera data processing…

� Add camera and other media support nodes� Add Watson Cognitive Computing services to process a data from

camera

Bluemix and Node-RED

20

…camera data processing

25/04/2018 TIES4371 - Lecture 5

UNIVERSITY OF JYVÄSKYLÄ

Watson Cognitive Computing services on Bluemix

Bluemix and Node-RED

21

Watson Visual Recognition service helps to find meaning in visual content and develop smartapplications that analyze the visual content of images or video frames to understand what ishappening in a scene. (https://console.bluemix.net/catalog/services/visual-recognition)� analyze images for scenes, objects, faces, and other content.� choose a default model off the shelf, or create your own custom classifier.� find similar images within a collection.� etc.Watson Text to Speech service processes text and natural language to generate synthesizedaudio output. (https://console.bluemix.net/catalog/services/text-to-speech)

25/04/2018 TIES4371 - Lecture 5

…camera data processing

UNIVERSITY OF JYVÄSKYLÄ

Watson Cognitive Computing services

Bluemix and Node-RED

22

� Create instance of the Watson Visual Recognition service and create new credentials…� Write down the ‘api_key ’ value…

25/04/2018 TIES4371 - Lecture 5

…camera data processing

UNIVERSITY OF JYVÄSKYLÄ

Watson Cognitive Computing services

Bluemix and Node-RED

23

� Create instance of the Watson Text to Speech service and create new credentials…� Write down the ‘username ’ and ‘password ’ value…

25/04/2018 TIES4371 - Lecture 5

…camera data processing

UNIVERSITY OF JYVÄSKYLÄ

Add a new Node-RED flow

Bluemix and Node-RED

24

� Login to IBM Bluemix and go to the list of your application on a Dashboard:https://console.bluemix.net/dashboard/apps

� Find your Node-RED application created in the previous example and go to its URL to access the Node-REDflow editor.

� Create a new Node-RED flow by clicking the ‘+’ icon

25/04/2018 TIES4371 - Lecture 5

…camera data processing

UNIVERSITY OF JYVÄSKYLÄ

Add new components to Node-RED environmentthat allow to browse and send files, or access Web Camerafrom the Node-RED application

Bluemix and Node-RED

25

� Add new components (node-red-contrib-browser-utils and node-red-contrib-play-audio ) to Node-RED palette via palette manager

25/04/2018 TIES4371 - Lecture 5

…camera data processing

Add Watson services to the flow� Locate the ‘Visual Recognition ’ and ‘Text to Speech ’ node under the

‘Watson_services ’ section in the palette window.� Drag both these nodes onto the workspace and set corresponding credentials

(‘api_key ’ ,or ’username ‘ and ‘password ’) via node configuration.� Also select ‘Detect: Classify an image’ for Visual Recognition services.

UNIVERSITY OF JYVÄSKYLÄ

Build the final flow for visual recognition app

Bluemix and Node-RED

26

� Locate the ‘File Inject ’ and ‘Camera ’ nodes under the ‘Inputs ’ section, and ‘Function ’ node� under the ‘Function’ section in the palette window, and drag them onto the workspace and connect the

nodes together as shown below.

25/04/2018 TIES4371 - Lecture 5

…camera data processing

UNIVERSITY OF JYVÄSKYLÄ

Add logic to the flow

Bluemix and Node-RED

27

� Configure the ‘Function ’ node with the following code…

� Now the flow is ready to be tested. Click ‘Deploy’ to deploy it.

Having a Web Camera, you can click the button on the Camera node to take a picture and have Watson classify itfor you and speak out the result. (You might have to give the permissions on your browser to access the camera).

Other option is to inject an image file to the flow. Prepare some picture and use button on the File Inject node tobrowse image and send it to the flow.

25/04/2018 TIES4371 - Lecture 5

…camera data processing