Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace

1 Intelligent Knowledge-as-a-Service

IoT is a King, Big data is a Queen and Cloud is a Palace

Abdur Rahim Innotec21 GmbH, Germany

Create-Net, Italy [email protected]

Acknowledgements- iKaaS Partners (KDDI and other

partnes)

2

§  Motivation § Convergence/opportunities/applications

§  Challenges and requirements §  Convergence approach §  iKaaS EU-Japan project §  Conclusion

Outline

3

Convergence of Technologies

Source-IDC

4

Where is the value of IoT?

§  In the past, connec1vity and number of the devices were the main driver of IoT

§  Data is nothing without big business value insight §  IoT without BIG DATA is first genera1on of IoT

5

The real value is not just sheer number of connected devices and data

§  The real opportunity is improved business value-‐new revenue models, lower cost, improved client experience, beGer insight improve outcomes

Source-IDC

6

Big data-how we understand it

Source: Nenad

7

IoT in BIG data

§  IoT presents challenges in combina1on of all BIG data characteris1cs (3Vs/4Vs)

§  Most challenging IoT applica1ons match with either or both Velocity & Volume and some1mes also Variety (situa1on and context) §  Velocity driven-‐applica1on

§  A wearable sensor produces about 55 million data points pro day (challenge for storage), whereas some medical wearable's (like ECG) produce up to 1000 events per second (challenge for real-‐1me processing)

§  Volume driven-‐applica1ons §  GE each day gathers 50 million pieces of data from 10 million sensors, off equipment worth $1 trillion

8

Typical IoT applications

Source: Harvad business review

9

IoT BIG data applications

§  Massive monitoring/Deep understanding (observe of behavior of

many thing””, gain important insight §  Health example (understanding the cause of diseases/comorbidi1es/indicators)

§  Real-‐1me ac1onable insight (Real-‐1me analy1c, detect and react in real-‐1me) §  Health example (real-‐1me fall detec1on and poten1al reac1on for aging popula1on)

§  Performance op1miza1on (configura1on, energy, health-‐care) §  Health example (Improve overall healthcare efficiency)

§  Proac1ve and predic1ve func1onal applica1ons §  Health example (proac1ve and predic1on iden1fica1on of diagnos1c in healthcare applica1ons (before thing occur)

10

Tradi9onal methods IoT/Big data

Centralize Distributed

More power More machines

Summarize data Keep all data

Transform and store Transform on demand

Pre-‐define schema Flexible/no-‐schema

Move data toward compute Move compute towards data

Less data/more complex algorithms More data/simple algorithms

Philosophical differences of Big data analytic

11

§  More distributed processing and storage of the massive data as well as cloud func1onali1es §  Processing capabili1es and data posi1oned closer to users §  Distributed and/or edge compu1ng and processing of IoT big data §  Distributed storage

§  Virtualiza9on of IoT devices: Access to advanced resources/specialized hardware, including GPUs, sensors, etc.

§  Interoperability: Interoperability between cloud/IoT services and infrastructure

§  Accountability-‐ Services and data hosted and executed across borders §  Elas1city and scalability of cloud data management

§  Security and privacy §  Data integrity, localisa1on, and confiden1ality §  Data localiza1on is one of the biggest challenges §  Security and privacy-‐by-‐design (across value chains including SLAs,

sogware algorithms and new data management models

IoT big data and cloud challenges and requirements

12

IoT Big data platform requirements

Intelligent and dynamic

Scalable

Real-time Unified view Distributed

Security and privacy

13

What cloud offers?

§  Dynamic and flexible resources sharing plaiorm §  Offers scalable, elas1city resources and data management §  Loca1on independent can be access from any where §  Reliable and easy access of the services §  Large amount of compu1ng and storage resources §  It is also more homogeneous (unified)

14

Convergence of IoT-Big data and Cloud

"Cloud compu*ng a new business model and management (e.g. data and device) paradigm of Internet of thing and Big data" ”IoT Big data is to enlarge the opportuni*es of cloud service provisioning §  Convergence Approaches

§  Centralize approach (Bring IoT func1onali1es in Cloud) §  Distribute approach (Bring Cloud func1onali1es in IoT)

15

IoT-Big data-Cloud: Centralize approach

§  Bring IoT data in the cloud §  Processing and compu1ng the data and deploy management

tools in cloud §  This approach this good if service are provided among objects

located in mul1ple loca1on

IoT$Cloud$Pla+orm$

hosting databases applications partners SI

All devices Our$managed$devices$ your$devices$

Cogni2ve$capability$

16

IoT-Big data- Cloud: Distributed approach

§  Edge/fog compu1ng-‐Stream Processing and storage of data close to users/near to devices §  To distribute data to move it closer to the end-‐users to eliminate latency, numerous hop, and support mobile compu1ng and data streaming

§  Usability §  High-‐latency and real-‐1me ac1onable insight (the data flow to fast to be processed)

§  Data/intelligence context are geographically distributed §  The datasets have strict privacy, security and regula1on constraints that prohibit their transfer outside of the paten domain

§  Domain specific service and applica1ons

iKaaS (H2020 EU-‐Japan) IoT-‐Big and Cloud Project

17

The goal of iKaaS project is to combine ubiquitous and heterogeneous sensing, seman1c, big data and cloud compu7ng technologies in a plaiorm enabling the Internet of Things distributed process consis1ng of con1nuous itera1ons on data inges1on, data storage, analy7cs, knowledge genera7on and knowledge sharing phases, as founda1on for cross-‐border informa1on service provision.

18

Project objec1ve

Architecture framework (Distributed)

19

Local CloudLocal Cloud

KaaS

App.App.

Sensors/IoT Devices

Sensors/IoT Devices

Storage StorageData

Query

QueryQuery

SecurityGW

Data

Data Data

SecurityGW

Global Cloud

Knowledge Knowledge

Storage

20

iKaaS

ProgramableService logic

Publish sensor needs, Privacy needs, RT needs, Reliability needs (constraints)

Alloca1on op1mizer

Alloca1on decision

Cloud, data center

Cloud Controler

Move to the local Cloud A

… or stay in local Cloud

Service and processing migra1on

Cloud, data center

Move to the Global Cloud B

… or stay in local Cloud

•  Smart service logic –  Autonomously analyse applica1on requirements, user preferences –  Register the services/deployment of services

•  Alloca9on manager –  The most appropriate deployment of service must achieve the best

balance among cloud resources, system performance, quality of service and cost.

–  Appropriate service execu1on •  Service/task Manager (Query, control, and reconfigura1on)

–  Analysis of the applica1on request(s) using iKaaS service model/templates; flexible/autonomic selec1on of more appropriate cloud resources

–  Reconfigure the service logic on run-‐1me (e.g, dynamically changes the services/business logic)

–  Synchroniza1on of the service logic deployment, service migra1on, decision between local and global cloud

Service deployment and orchestra1on

Distributed execu1on environment

22

Service Query

Service query (Query control)

service logic

Smart logic

Local Cloud

Glob

al Cloud

Configura1on manager

Configura1on and alloca1on

Manager

Depe

nden

t Inde

pend

ent

Migra9o

n

Synchroniza9on

Programmable applica1on logic

Service catalogue

Service Catalogue

Service/task M

anager

Service/task M

anager

23

Multi-scale service migration

§  Migra1on of rela1onship logic to local cloud

Service request

Service component migration

Service results

Service orchestration

Service o

rche

stratio

n

Local Cloud

Service execution

Local Cloud

Service execution

Local Cloud

Service execution

uCore Framework

Analysis

Decision Making

Service Logic description

Monitoring

Learning

Service component results

Cognitive Engine

Service and associated meta-‐data

Global Cloud

Smart Virtual ObjectsComputing in the

Global Cloud

Mul1-‐scale applica1on migra1on

My laptop

Gateway1

Server applica1on

iKaaS Component

Temp. sensor

Gateway2 Temp. sensor

Gateway3

Temp. sensor

1ms readings

1ms readings

1ms readings

iKaaS Component

Local Proc.

Local Proc.

Local Proc.

Daily computa1on results

Final result

Service migra1on

Service migra1on

In red: applica1on logic deployment In blue: data gathering and consolida1ng

• Applica1on’s logic can be migrated near the data sources • mul1-‐scale (recursive) process: the applica1on’s logic can be broken down again and further migrated

Security Gateway

Global Cloud

Local Cloud

Privacy Policy

Security Policy

Global Cloud

Security Gateway (2)

•  Security and Privacy by Design Concept •  Main Func1ons:

–  Policy Management & Nego1a1on (Cross-‐Border) –  Authen1ca1on and Access Control (Service Level) –  Transforma1on of Data (Privacy Preserving Way)

•  Applica1on to Cross-‐border Scenario

26

Security GW

Local App.

Security GW

Internal Use

External App.

Cross-‐Border Use

Local Cloud

Data Transfer

Policy Nego1a1on

Local Cloud

Design of the Security Gateway


Privacy Certificate DB

Privacy CA

Local Cloud

Local Cloud DBs Local Query Controller

Privacy Policy DB

Policy DBs Security Gateway

Token DB Security Policy DB

Global Cloud

Privacy Control Functions

Cache Policy DB

Owner DB

Key DB

Access Control Functions

Cache DB

Cache Manager

Query Control Functions

Data Processing Functions Global Platform

Application

28

§  Procedure §  Token Issuance

§  I. An applica1on requests the privacy CA to issue the privacy cer1ficate. §  II. The applica1on searches the security gateway of the domain where there are the local cloud DBs suited for the objec1ve with using the

query control func1ons on the global plaiorm. §  III. The applica1on calls func1on Issuance of Token that the security gateway provides. The applica1on then specifies the DB IDs of the local

cloud DBs that it wants access to, and sends the privacy cer1ficate. §  IV. The security gateway confirms the values of parameters CA Domain Name and Expires listed on the privacy cer1ficate to verify the

correctness of the cer1ficate. §  V. The security gateway checks the values of Applica1on IP, LC Domain Names and LC DB IDs listed on the privacy cer1ficate to validate the

applica1on and the request. §  VI. The security gateway creates a token and returns it to the applica1on.

§  Data Request §  I. An applica1on generates the MAC of the SGW-‐query with using the token, which is a common key. §  II. The applica1on calls func1on Getng Data that the security gateway provides and transmits SGW-‐query and the MAC to the security

gateway. §  III. The security gateway extracts the corresponding token from the token DB with the values of the Applica1on ID and Applica1on IP headers

and checks the expired date of the token. §  IV. The security gateway generates the MAC from the token and the SGW-‐query to verify the authen1city of the query. The value of the Time

Stamp header is also confirmed. §  V. The security gateway transmits the LCD-‐query to the local query controller. §  VI. When the data are returned from the local cloud DBs, the security gateway confirms the privacy type of the DBs while searching the token

DB. §  VII. If the data stored in the non-‐privacy DB are returned, the security gateway returns the data to the applica1on without doing anything.

Otherwise, Steps 8-‐-‐11 are carried out. §  VIII. The security gateway extracts the corresponding owner IDs from the owner DB with using the value of the Owner AGributes header. §  IX. The security gateway searches the privacy policy with using the extracted owner IDs and the values of the Applica1on ID and LC DB IDs

headers and confirms the status of the consent of the corresponding data owners. §  X. The security gateway extracts the data such that the data owner agrees on the transfer and returns the extracted data to the applica1on.


•  Example of Security Policy –  Token Configura1on (such as period and accessible informa1on)

should be defined for each applica1on category and country of the domain that applica1on is executed.

Level DB 1 DB 2 ・・・ DB N

Applica1

on A

Administrator 1 1

UK 0 / JP 2mo

Non-‐Privacy

UK 3h / JP 3h

Non-‐privacy ・・・ UK 0 / JP 0

Privacy

Administrator 2 2

UK 1h / JP 2h

Privacy

UK 5h / JP 0 Non-‐privacy ・・・

UK 0 / JP 0 Privacy

呍呍呍

呍呍呍

呍呍呍

呍呍呍

呍呍呍

Administrator M M UK 0 / JP 0

Non-‐privacy UK 1h / JP 0 Non-‐privacy ・・・

UK 0 / JP 0 Privacy


30

§  Performance Evalua1on Results

§  Transac1on 1me of data collec1on is prac1cal. §  Cache func1on is effec1ve for reducing the transac1on 1me.

Security Gateway(6)

# of Data Non-‐Private Private Using Cache Func.

1000 16.868171 215.650792 3.426036

10000 57.940439 254.608338 5.528918

100000 504.188900 776.667116 21.692454

1000000 5109.974000 5872.079780 155.043988

31

Take away message

§  Convergence is everywhere §  If you start innovation think on the how your

business will convergence and scale §  When we talk IoT, it is actually the large-

scale §  NEED of large-scale IoT is to exploit Big data

for smart IoT services that processed and executed on the cloud to derive business value insight

Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace

Data & Analytics

Transcript of Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace