IOT BIG DATA: REALISING THE FULL POTENTIAL OF …...POTENTIAL OF THE IoT ETSI Presentation November...
Transcript of IOT BIG DATA: REALISING THE FULL POTENTIAL OF …...POTENTIAL OF THE IoT ETSI Presentation November...
IOT BIG DATA: REALISING THE FULL POTENTIAL OF THE IoTETSI PresentationNovember 2016
THE IOT BIG DATA ECOSYSTEM
IOT TODAY
Disparate systemsSilos of heterogeneous dataValuable data, proprietary implementationsWidespread innovation by device manufacturers
VISION FOR IOT
Sharing of dataStandardisationData monetisationIncreased innovation across all tiersIncreased protections: privacy, security etc
VISION FOR AN IOT BIG DATA ECOSYSTEM
Creation of new business models, products and services by applying various capabilities, such as mash-up and analytics, to IoT and context data captured from multiple sources“
“- Enable an ecosystem through which data can be harmonised and shared from
multiple sources
- Develop new commercial products and services by applying capabilities to harmonised IoT and context data
“Water, water, every where,Nor any drop to drink.”
Expect we will be flooded in data but how usable will it be?
Some of the challenges Device protocols/ APIs Context data protocols (JSON/
CSV/ XML etc) Knowing the accuracy of devices Consistency of units Device reliability Device stability Finding the data
4
THE CHALLENGE OF DATA SET DIVERSITY
5
Source: Internet of Things Study 2016, Evans Data Corporation
IOT DEVELOPER INSIGHTS
KEY AREAS OF GSMA FOCUS 2016/2017
Standardisation - data & APIs Data privacy Ecosystem collaboration and
innovation
Agreement of de-facto standards: Harmonised data sets for key
market sectors Common framework for delivery
of IoT Big Data ecosystem
Common API for sharing of harmonised data & control
Directory of harmonised operator IoT Big Data
GSMA position on Privacy and Big Data
Big Data Guidelines incorporating IoT Big Data published
Assessment of candidate API technologies
Ecosystem input into de facto standards
Innovation and showcases
7
IOT BIG DATA USE CASES
Smart Home Utility metering Home appliance
Agriculture Farms Greenhouses
Environment Air quality Water quality Weather
Smart Cities Roads Parking
Automotive Vehicle Faults
The potential scale of data being stored, processed or queried given
The number of IoT devices and update frequency
Combining current / historical IoT data with external data
Managing privacy issues including consent, aggregation, anonymisation
The large variety of devices in the field where each vendor/ device type might have its own specific interface
Differences in device attributes – names, units etc - even where the device is reasonably generic
8
PARTICULAR CHALLENGES FOR IOT/BIG DATA
GSMA has published three technical documents defining the approach for delivery of IoT Big Data services to the general third party application developer ecosystem1. Technical Architecture Framework2. API Specification for Data & Control exposure towards 3. Initial set of Harmonised Entity Definitions
Documents available from http://www.gsma.com/connectedliving/iot-big-data/
9
GSMA Technical Documents
TECHNICAL ARCHITECTURE FRAMEWORK
Logical architecture provides flexibility so organisations can follow different business strategies‒ Analytics on the left‒ Direct data & control on the
right Common elements:
‒ Harmonised data entity definitions
‒ Common APIs for direct data & control
FIT WITH ONEM2M
Initial proprietary interfaces (I2) will be augmented with OneM2M
Substantial simplification of interfacing layers
Gateways handle local data acquisition & control
Dedicated IoT applications delivered using oneM2M stack
Hybrid architecture supports richer applications stack including advanced analytics, visualisation, machine learning
Applications which span broader set of devices and context data
Data transformation/ storage to be used in rich analytics/ machine learning
AE
CSE
NSE
Mca
Mcn
oneM2M Functional
Architecture
I2 = Mca
FIT WITH FIWARE CONTEXT BROKER
FIWARE (Telefonica, Orange, NEC and others) have specified NGSIv2
FIWARE have also implemented a generic ’context broker’ component ‘Orion’ which can be used for Big Data service delivery
Flexible environment for communicating harmonised entity data throughout the architecture
13
HTTP Verb: POSTResource URL: /v2/entities?options=options
Request body example{
"id": "a01b4849-2eca-4518-b5a3-230f9749147e","type": "AgriSoil","source": "www.samplefarmproduct.com","dataProvider": "OperatorA","name": "Sand"
}
EXAMPLE NGSIv2 – CREATE ENTITY
Create an entity including one or more attributes;
Retrieve entity (/specified attribute`s) by entity identifier;
Update one or more attributes of an existing identity;
Remove an entity; Get an attribute value; Update an attribute value; Remove an attribute of an entity; List entities matching specified criteria;
List known entity types; Retrieve entity type information; Create a subscription; List subscriptions; Retrieve details of a subscription; Update details of a subscription; Delete a subscription; Batch update; Batch query.
14
NGSIv2 OPERATIONS
15
Agriculture AgriCrop
AgriGreenHouse
AgriParcel
AgriParcelRecord
AgriParcelOperation
AgriPest
AgriProduct
AgriProductType
AgriSoil
HARMONISED DATA ENTITIES
Environment AirQuality
EnvironmentObserved
PointOfInterest
WaterQuality
WeatherForecast
WeatherObserved
Smart Home Building
BuildingType
BuildingRecord
BuildingRecordType
Connected Car /Smart Cities
Vehicle
VehicleType
VehicleFault
Road
RoadSegment
General IoT Device
DeviceRecord
DeviceRecordType
Machine
MachineType
Subscriber
SubscriptionService
Entity definitions published at : https://github.com/GSMADeveloper/HarmonisedEntityDefinitions
16
EXAMPLE ENTITY DEFINITION - AGRISOIL
Attribute Name Attribute Type
Description Mandatory/Optional
May be Null
name Text The name of this soil type. M N
alternateName Text Alternative name of this soil type. O Y
description Text A description of this soil. O Y
refAgriProduct Array of Reference
An array containing a JSON encoded sequence of characters referencing the unique ids of the recommended AgriProduct fertiliser (or other) product(s).
O Y
Attribute Name Attribute Type
Description Mandatory/Optional
May be Null
id Text Unique id of this instance of this entity. M N
type Text Must be equal to "AgriSoil". M N
dateCreated DateTime Entity creation timestamp. M N
dateModified DateTime Timestamp of the last modification of the entity.
M Y
source Text A sequence of characters giving the source of the entity data as a URL.
M Y
dataProvider Text A sequence of characters identifying the originator of the harmonised entity.
M Y
<AgriSoil><Generic Attributes>
<AgriSoil><Entity Specific Attributes>
17
EXAMPLE ENTITY CODING (JSON) - AGRISOIL{"id": "789363b4-c771-43d6-8505-ca582efe8fcd","type": "AgriSoil","dateCreated": {"value": "2016-08-08T10:18:16Z", "type": "DateTime"
},"dateModified": {"value": "2016-08-08T10:18:16Z", "type": "DateTime"
},"source": {"value": "http://www.samplefarmproduct.com", "type": "URL"
},"dataProvider": {"value": "OperatorA", "type": "Text"
},"name": {"value": "Clay", "type": "Text"
},"alternateName": {"value": "Heavy soil", "type": "Text"
},"description": {"value": "Fine grained, poor draining soil. Particle size less than
0.002mm", "type": "Text"},"refAgriProduct": ["ea54eedf-d5a7-4e44-bddd-50e9935237c0","275b4c08-5e52-4bb7-8523-74ce5d0007de"
]}
Analytics & Visualisation Better understanding of data relationships
E.g. weather to crop yields Traffic to pollution
Acting on that information Decision to harvest crops based on
actual/ forecast weather Changing traffic signalling to improve
pollution levels
Machine Learning Use of AI/ Deep Learning
Establishing new/ non obvious relationships between data and outcomes
Advanced image/ video processing applications e.g. traffic monitoring, smart parking
18
NEXT STEPS FOR IOT BIG DATA