EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP...

53
Brewing beer with Python Chesco Igual @chescales

Transcript of EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP...

Page 1: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Brewing beer with PythonChesco Igual@chescales

Page 2: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

What shalt thou learn?

- Building an IoT backend -

- Technologies, Protocols and Tools -

- Backend considerations -

- Meet a full running architecture -

- Learn to brew beer -

Page 3: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

What shalt thou learn?

- Building an IoT backend -

- Technologies, Protocols and Tools -

- Backend considerations -

- Meet a full running architecture -

- Learn to brew beer -

Page 4: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

MiniBrew

Page 5: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

So what is it?

Page 6: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

So what is it?

Yay!! Awesome!!

(and others)

Page 7: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

So what is it?

Yay!! Awesome!!

(and others)

Page 8: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Let’s go technical

Page 9: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

Real-time data

Page 10: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

Security - Real-time data -

Page 11: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

Obfuscation - Real-time data -

- Security -

Page 12: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

Authentication - Real-time data -

- Security -

- Obfuscation -

Page 13: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

Two-waycommunication

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

Page 14: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

Resiliency - Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

Page 15: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

Lightweight - Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

Page 16: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Project Requirements

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

- Lightweight -

- Last known status -

- Debugging -

- Admin site -

- Mobile app api -

- rainbows, etc. -

Page 17: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

What thou shalt take care about too...

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

Page 18: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

What thou shalt take care about too...

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

Page 19: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

when I read the project specs for the first time

Source: thecodinglove.com

Page 20: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Step by Step

Page 21: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Communications Protocol

Page 22: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Communication Protocols

HTTP

AMQP

XMPP

CoAPMQTT

DDS

Page 23: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Communication Protocols

HTTP

AMQP

XMPP

CoAPMQTT

DDS

Page 24: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Comprehensive IoT Backend Solutions

Page 25: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Amazon IoT

Page 26: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time
Page 27: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

So then…?

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

- Lightweight -

- Last known status -

- Debugging -

- Admin site -

- Mobile app api -

- rainbows, etc. -

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

PROJECT REQUIREMENTS EXTRAS

Page 28: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Let’s set up our own(get a broker…)

Page 29: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Options?

Page 30: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Options?Top player for many years

Scalability proven (vertical and horizontal)

Can convert from MQTT to other protocols (AMQP)

No payment per use

Familiarity

Page 31: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Extra bonus!

???

Page 32: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Extra bonus!

Page 33: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Now what?Let’s talk to that broker

Page 34: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Eclipse library

https://github.com/eclipse/paho.mqtt.python

Page 35: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

NO CODE

Page 36: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time
Page 37: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

API time

Page 38: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Again, options?

Page 39: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

How are we doing?

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

- Lightweight -

- Last known status -

- Debugging -

- Admin site -

- Mobile app api -

- rainbows, etc. -

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

PROJECT REQUIREMENTS EXTRAS

Page 40: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Authentication

Page 41: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Let Python decidehttps://github.com/rabbitmq/rabbitmq-auth-backend-http

Page 42: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

How are we doing?

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

- Lightweight -

- Last known status -

- Debugging -

- Admin site -

- Mobile app api -

- rainbows, etc. -

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

PROJECT REQUIREMENTS EXTRAS

Page 43: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

How are we doing?

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

- Lightweight -

- Last known status -

- Debugging -

- Admin site -

- Mobile app api -

- rainbows, etc. -

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

PROJECT REQUIREMENTS EXTRAS

Page 44: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Obfuscated andLightweight messages

Page 45: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Protocol Buffershttps://github.com/google/protobuf

Page 46: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Protocol Buffershttps://github.com/google/protobuf

Page 47: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time
Page 48: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

How are we doing?

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

- Lightweight -

- Last known status -

- Debugging -

- Admin site -

- Mobile app api -

- rainbows, etc. -

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

PROJECT REQUIREMENTS EXTRAS

Page 49: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

How are we doing?

- Real-time data -

- Security -

- Obfuscation -

- Authentication -

- 2-way communication -

- Resiliency -

- Lightweight -

- Last known status -

- Debugging -

- Admin site -

- Mobile app api -

- rainbows, etc. -

- Scalability -

- Proven technologies -

- small tech stack -

- Error tracking -

- Reduce data transfer -

- Documentation -

PROJECT REQUIREMENTS EXTRAS

Page 50: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Final Architecture

Page 51: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Wanna roll with us?Barcelona (ES) Almere (NL)

elements.nl/careers

Page 52: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Thank You

Page 53: EuroPython 2016 - @chescales Chesco Igual...MQTT CoAP DDS Communication Protocols HTTP AMQP XMPP MQTT CoAP DDS Comprehensive IoT Backend Solutions Amazon IoT So then…? - Real-time

Questions?