Einsatz von Big Data Technologien im Geo-Marketing · 21/9/2017 · •Umfassende Data Management...

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Einsatz von Big Data Technologienim Geo-MarketingDDS Data Days 2017

Hans ViehmannPorduct Manager EMEA

Heidelberg, September 21, 2017

Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

Marketingim Wandel

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 3

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 4

DISCOVERWEB

PRINT

DIGITAL ADS

WORD OF MOUTH

SOCIAL

TV

NEED

RESEARCH

RETAIL STORE

WEB SHOP

REVIEWS

SEARCHKW/ADS

RECEIVEOFFER

EMAIL

CONTACT CENTER

WEB SHOP

TRACK ORDER

SHOP AND BUY

RECEIVE PACKAGE

MISSINGITEM

WEB SHOP

CONTACT CENTER

SUPPORT PORTAL

SETUPPHONE

POSTREVIEW

JOINGROUPS

NETWORKISSUE

PHONEDAMAGE

TERMINATESERVICE

BILLINGISSUE

SOCIAL

RETAIL STORE

SOCIAL

MAKEPAYMENT

CONTACT CENTER

SUPPORT PORTAL

CHANGEADDRESS

CONTACT CENTER

RECEIVEOFFER

RESTARTSERVICE

CONTACT CENTER

RETAIL STORE

WORD OF MOUTH

SOCIAL

EMAIL

REFERFRIENDS

BRANDEDCOMMUNITY

Mehr InteraktionMehr Daten

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 4


Technologische LösungsansätzeDrei Möglichkeiten

1. Einsatz „intelligenterer“ Software

2. Umstellung auf leistungsfähigere Hardware

3. Vollständig neue Architektur

5


Technologische Lösungsansätze (2017)Drei Möglichkeiten


2. Umstellung auf leistungsfähigere Hardware in der Cloud


6


Einsatz „intelligenterer“ SoftwareNicht jedes Problem erfordert eine neue Architektur

7


Anwendungsbeispiel Geomarketing

• Vorbereitung von Massen-Mailing im Handel

• Zuordnung von ca. 1 Mio. Adressdatensätzen zur nächstgelegenen Filiale

• Bestimmung nach Fahrstrecke, nicht Luftlinie

8

Filialstandort

Wohnort Kunde


Anwendungsbeispiel GeomarketingMögliche Lösungsansätze

• Berechnung der Fahrstrecke jedes Kunden zu jeder Filiale

– Aufwändige Berechnung, erfordert auch bei guter Parallelisierung viel Zeit

• Aufbau von Voronoi-Diagrammen zur Gruppierung von Kunden und Filialen

– Vorverarbeitung zeitintensiv

• Geschickte Ausnutzung von Netzwerk-Datenmodell in der Datenbank– Bestimmung von Straßensegment aus Geocoding

– Berechnung aller Straßensegmente im Einzugsgebiet der jeweiligen Filiale

– Verknüpfung von Kunde und Filiale in einer einzigen SQL-Abfrage

9


Anwendungsbeispiel GeomarketingVorgehensweise

• Straßennetz in der Datenbank

– In diesem Falle HERE Daten für die USA im Oracle Data Format (ODF)

• Verwendung des selben Straßennetzes für Geocoding und Routing

– Damit ID des Straßensegments und ID der zugehörigen Kante im Router übereinstimmt

• Ausnutzung von Parallelisierung beim Geocoding– Bulk Geocoding mittels Parallel Pipelined Table Function

• Umgebung als VirtualBox Image für Workshops verfügbar

– samt detaillierter Anleitung

10


Umstellung auf leistungsfähigere Hardware in der Cloud

11


Massiver Wandel bei CX Plattformen

• Verbindung von eigenen Systemen mit Cloud-basierten Services

– Neue Projekte fast ausschließlich in der Cloud

• Zusammenführung eigener Daten mit Daten von Partnern und Datendienstleistern

• Umfassende Data Management Plattformen verfügbar

–Wesentlicher Bestandteil des Oracle CX Ökosystems

– Zusammenführung der Kundenhistorie mit Verhaltensdaten

– Pool von ca. 5 Mrd. anonymen Profilen, verknüpft über ID Graph

• Cloud-basiertes Data Warehouse oder Big Data Platform naheliegend

– „Elastische“ Kapazität, reduzierte Kosten, kürzere time-to-market

12


Vollständig andere ArchitekturAusnutzung von Big Data Technologien

13


Addressing Big Data Challenges

VOLUME VELOCITY VARIETY VALUE

SOCIAL

BLOG

SMARTMETER

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

4th Generation Data Architecture for Big Data

WarehouseData FactoryData Lake

Events and Streaming

Data Platform

Data Discovery Lab

AnalyticTools

Execution

Innovation

ActionableInformation

ActionableInsights

ActionableEvents

ActionableDiscoveries

Enterprise Data

Other Data Sources

Data Streams

BusinessData

Social/LogData


Using Hadoop to Address Big Data Challenges

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. Hadoop is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.


Benefit of Big Data Technologies

• Low cost and high horizontal scalability infrastructure

• Allowing storage of more data, more details over longer time periods

• Cost-effective way to analyse huge amounts of data

• Dealing with variable data by means of „schema-on-read“ capability

• Complementary to existing data warehouse technologies

17


• Database technologies

– Designed for „high-density“ data

– Transactionality and data validation

– High availability

– Comprehensive security and access control

– Existing tooling and workflows

–Operational experience

• Big Data Technologies

– Designed for „sparse“ data

– Low cost and high horizontal scalability

– Allowing storage of more data, more details, over longer time periods

– Cost-effective way to analyze huge amounts of data

– Rapid ingestion, not necessarily consistent

– „Schema-on-read“ capability for variable data

18

Differentiation against conventional databases


Working with geospatial data on Big Data platforms

• Requires support for geometric functions, topological operations

– eg. determining the centroid

– Needs to deal with projections, complex operations such as buffering, ...

• More complex processing usually requires spatial indexing

– eg. spatial joins

• Spatial data usually comes in specific formats (Shapefiles, GeoJSON, ...)

• Needs to cope with location information which is only included implicitly

– requires geo-enrichment

• Visualization is very valuable for inspection of source data and results

19

Location Intelligence has specific requirements


Categorization and filtering based on location and proximity

Preparation, validation and cleansing of Spatial and Raster data

Visualizing and displaying results on a map

Spatial querying and analysis of Hadoop data with SQL

What problems can Big Data Spatial analysis address?

Data Harmonization using any location attribute (address, postal code, lat/long, placename, etc).


Data enrichment service API using GeoNames and geometry hierarchy data

SPARK and MapReduceroutines for distance calculations, PointInPolygon, buffer creation, Categorization, KMeansClustering, Binning

Spatial processing using of data stored in HDFS or NoSQL. Raster processing operations: Mosaic and sub-set operations. Geodetic and Cartesian data

HTML5 Map Visualization API

Hive SQL APIQuery from Oracle DB with Big Data SQL & Oracle SQL Connectors for Hadoop

Big Data Spatial Features


Example: Development of new insurance products

• Analyzing customer behaviour in Data Discovery Lab

• Using data from in-vehicle telematics box

• Exploratory stage

– Determining data quality

– Interpreting location data

–Merging data with customer profile date from other internal sources

– Developing possible factors for premiums

• Next step: converting to production workflow

– Automated data processing and derivation of actionable results

22

Pay-as-you-drive model at Italian insurance company


Related topic: Graph analysis

• Graphs are probably familiar from network models

– Reachability

– Shortest Path (eg. Dijkstra algorithm)

• Concept is increasingly popular and widely applicable–Modeling entities as vertices

–Modeling relationships as edges

• Graph algorithms enable topological analysis

– Detecting behavioural patterns

– Creating knowledge graphs, potentially in conjunction with Machine Learning

23


Examples for Graph Analytics

• Community detection and influencer analysis

– Churn risk analysis/targeted marketing, HR Turnover analysis

• Product recommendation

– Collaborative filtering, clustering

• Anomaly detection

– Social Network Analysis (spam detection), fraud detection in healthcare

• Path analysis and reachability

– Outage analysis in utilities networks, vulnerability analysis in IP networks, „Panama Papers“

• Pattern matching

– Tax fraud detection, data extraction

24


Trend: Machine Learning replacing Big Data

• KDnuggets studyin April 2017

– Google trends analysis

26

Source: Geospatialmedia.net


Technologische Lösungsansätze (2017)Drei Möglichkeiten


2. Umstellung auf leistungsfähigere Hardware in der Cloud


27


Wirklich eine vollständig neue Architektur?SO

UR

CES

DATA RESERVOIR DATA WAREHOUSE

Oracle Database

Oracle IndustryModels

Oracle Advanced Analytics

Oracle Spatial & Graph

Big Data Appliance

Apache Flume

OracleGoldenGate

Oracle Event Processing

Cloudera Hadoop

Oracle Big Data SQL

Oracle NoSQL

Oracle R Distribution

Oracle Big Data Spatial and Graph

Oracle Database

In-Memory, Multi-tenant

Oracle Industry Models

Oracle Advanced Analytics

Oracle Spatial & Graph

Exadata

OracleGoldenGate

Oracle EventProcessing

Oracle DataIntegrator

Oracle Big DataConnectors

Oracle DataIntegrator

28


Concluding remarks

29


The Forrester Wave™: Geospatial Analytics Tools And Platforms, Q3 2016

“While hardcore GIS professionals may start their work in other applications, when they want to solve spatial problems in production and with web- and IoT- scale data, Oracle gives themthe platform to do so.”

Confidential – Oracle Internal/Restricted/Highly Restricted 30

Analysts: Rowan Curran with Holger Kisker, Ph.D. and Emily Miller September 1, 2016

The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester Wave™ is a graphical representation of Forrester's call on a market and is plotted using a detailed spreadsheet with exposed scores, weightings, and comments. Forrester does not endorse any vendor, product, or service depicted in the Forrester Wave. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.


Interested in project experience, best practices, networking?Spatial and Graph Summit

• IOUG Business Intelligence, Warehousing and Analytics SIG have established annual BIWA Summit

– Planned for March 20-22, 2018 at OracleHQ, Silicon Valley, California

• Spatial and Graph Summit is separate track

– Lots of interesting material from previous years available on OTN

• Opportunity for interaction withSpatial and Graph product team

31


More resources

• Further information on oracle.com

– www.oracle.com/goto/spatial

• Blogs

– https://blogs.oracle.com/oraclespatial

• Developer forums on OTN

– https://community.oracle.com/community/database/oracle-database-options/spatial

• LinkedIn community

– „Oracle Spatial and Graph“ group

• Google+ community

– „Oracle Spatial and Graph SIG“

• Follow us on Twitter – @SpatialHannes, @agodfrin, @JeanIhm

32

http://www.oracle.com/goto/spatial

https://blogs.oracle.com/oraclespatial

https://community.oracle.com/community/database/oracle-database-options/spatial


Q&A

33


Cloud Computing enabling Big Data Analytics

• Scaleable resources readily available

• No setup cost

• No hardware investment

• Pay-as-you-go model

• e.g. Oracle Big Data Cloud Service

– Hadoop Platform, SPARK technology

– Data Integration

– R Distribution

–Oracle Big Data Spatial and Graph

35

Database

Backup

Big Data

General Purpose


Cloud Service – Management Console

Einsatz von Big Data Technologien im Geo-Marketing · 21/9/2017 · •Umfassende Data Management...

Documents

Transcript of Einsatz von Big Data Technologien im Geo-Marketing · 21/9/2017 · •Umfassende Data Management...