Mobile Database and Service Oriented Architecture
-
Upload
lucas-jellema -
Category
Technology
-
view
1.432 -
download
1
description
Transcript of Mobile Database and Service Oriented Architecture
1
MOBILE DATABASES AND SOALucas Jellema (AMIS, The Netherlands)
Workshop “Big Data and Its Impact” – 16 & 17 March 2013for the Military College of Telecommunication Engineering
2
THE PRESENTER:LUCAS JELLEMA• Lives in The Netherlands
(close to Amsterdam)• 1994-2002 Oracle Consulting Global Center
of Excellence for Internet Development• Joined AMIS in 2002 – now working as CTO,
Consultant (Architect & Technical Lead) & Trainer
• Oracle ACE (2005) & ACE Director (2006)• Author of ‘Oracle SOA Suite 11g Handbook’
(Oracle Press, 2010)• Presenter at Oracle OpenWorld, JavaOne,
OTN Yathra & more User Group Conferences• Frequent blogger at
http://technology.amis.nl• Active with SQL & PL/SQL, Java EE & ADF,
SOA, BPM & more Fusion Middleware
3
OVERVIEW
• Overview of mobile devices• Data is presented, collected and edited in
devices• Handle ‘challenged server data connection’• Introduce the Mobile Database • Mobile Database challenges
– Big Data, Fast Data, Fresh Data, Distributed Database System, Security, Notifications
• Overview of core SOA concepts and objectives
• How SOA helps to enable the Mobile Database
4
MY CAR – IS A DATA COLLECTOR
• Collecting metrics– Some for big data analysis for all cars (of this
type)– Some for car analysis of my car when serviced– Some for real time (traffic jam, local
temperatures)
5
FORMULA ONE TELE METRICS
6
CAR NAVIGATION SYSTEM
7
LIVE TRAFFIC INFORMATION
8
LIVE? TRAFFIC INFORMATION
9
ZOOMING TRAFFIC INFORMATION
10
LIVE ON-ROAD TRAFFIC INFORMATION SYSTEM == MOBILE DEVICE
11
ON SITE CRASH INSPECTION
12
PLETHORA OF MOBILE DEVICES & USAGES
13
MOBILE AGENTS
Sensors
Cars
Navigation Systems
Cameras
Tablets
Mobile Phones
Security Badge Scanners
Barcode Scanners &Handheld Devices
Push Buttons
Satellites
Report Read & Edit
Read & Refresh (Alert)
Microphones & Speakers
Security Gates
ATM
Audio GuideTraffic Information
14
DATA AND MOBILE DEVICES
• Some data requirements depend on the actual location of the device
• Some data has to be on the device even when no data connection is available
• Some data to be presented has to be fetched in (near) real time from a central service
• Some (big) data is collected for analysis at a much later point in time
• Some data is unstructured • Some data needs to be sent from the device
to a central service in real time• Most data will need to be refreshed at some
point in time – although the refresh rate differs
• Sometimes the user needs to be (very) aware when data is not recently refreshed
15
DATA USED ON MOBILE DEVICES
• Originates from/to be sent to a back end (“the server”)
• Through limited bandwidth• Sometimes entirely disconnected
– No reception– Not permitted to use device– Server side unavailable
16
NEED FOR ON-DEVICE-DATABASE
• Provide quick response for user• Work when off-line• Handle bursts of fast-data and piles of big
data• Take load off back end servers
17
REQUIREMENTS FOR MOBILE DATABASE• Able to handle right types of data – including
unstructured data• Easily accessible to apps
– Support for SQL • Right scalability, right performance• Right security
– Encrypted, Recoverable, Remote Wipe• Maintainable/upgradable
– Database software and database model– Pre-populated data (“firmware”)
• Able to fetch and process data from back-end– Compact, secure, smart (incremental data
updates)• Smart upload to server, also Fast and Big Data
– pre-aggregated, pre-filtered, compacted
18
TRANSPARENT TO USER AND APPS
• The local database should be ‘invisible’ to parties: they interact with data service to query or report data– However: it should be clear when data cannot
guaranteed to be fresh or complete
19
SECURITY
• Confidentiality – no unauthorized access– Encryption of data in storage and transport– Authorization on server access
• Integrity – no tampering, constraint compliant
• Availability – no data loss, right-time-access and freshness
20
HOW DOES DATA ENTER THE MOBILE DATABASE?• Initial load when app is first installed• Fetched from back end• Entered by mobile app(s)
• When– During install or upgrade of app– During use of the application (by the user or
device features such as GPS, Bar Code scanner, Camera or Recorder)
– When cache freshness is in doubt– When cache completeness
is in doubt– Synch upon re-connect:
submit collected new and changed data
21
SOME GENERIC CHARACTERISTICS
• Pre loading• Lazy loading (background)• Write behind (lazy posting)• Data Freshness and Completeness
management• Local fallback (next best thing)
22
DATA FRESHNESS AND COMPLETENESS MANAGEMENT• Based on actual time and geo-location…
– And possibly user, running app(s) and other context
• … assess if locally available data is fresh & complete
• When data is no longer fresh– Try to negotiate a smart refresh with the server– Depending on type of data: hide stale data,
show stale data with warning, continue to show stale data
• Incomplete data should be complemented– Different location/area, zoom level, …
23
PUSH NOTIFICATIONS
• The server side can process messages, updates and events from many directions
• Sometimes a mobile device should be informed of this new information
• The server needs to know about the relevance of information to a device (subscription)
• The server needs to be able topush information (or poll-basedsemi-push)
• The mobile database & app has to process the pushed information
push
24
MOBILE DATABASE(S) & BACK END SYSTEMS IS A …
25
DISTRIBUTED DATABASE SYSTEM
26
THE 12 OBJECTIVES FOR DISTRIBUTED DATABASE SYSTEMS
27
MOBILE DATABASES DO NOT SUPPORT DISTRIBUTED TRANSACTIONS
• Server side updates happen independent from clients– Refresh of Mobile Database may take place at
any point in the future• Data manipulation in Mobile Database is
committed locally on the device– Upload to server happens later: ‘write (far)
behind’• This means that at some point after the
transaction is complete, the transactionis ‘replicated’ to other databases
• During this replication,conflicts may occur
28
DATA REPLICATION – STAGE ONE
• Updates on the client are made through local apps an stored in the mobile database
• Local validations apply – based on the information available in the mobile device at the time of the transaction – Without consulting the back end
29
DATA REPLICATION (‘WRITE BEHIND’) CONFLICTS – STAGE TWO• Updates on client violate server side integrity
rules– Invalid data is not tolerated by the back end– The mobile client has previously accepted the
transaction – How is the app/user engaged to correct/resolve?
30
DATA REPLICATION (‘WRITE BEHIND’) CONFLICTS• Updates on client collide with changes in the
server– The client attempts to update a record that has
also been updated in some other way– How are conflicts detected?– How should conflicts be resolved?– How should parties be notified
about ‘lost conflicts’?
31
SOA = BAD
32
SOA =BusinessAgility through
Decoupling
33
DECOUPLING≈MANAGING DEPENDENCIES
minimize impact of change while maximizing reusability
34
COUPLING
A B
35
B
DECOUPLING
A B
36
B
DECOUPLING
A BC
37
B
ADDITIONAL DECOUPLING THROUGH INTERMEDIARY
A BCESB
38
B
COMPOSITE SERVICES
AD
E
ESB
39
B
TRANSFORMATION/ADAPTION OF PROTOCOL AND FORMAT
A
ESB
JSONHTTPSSL
SQLJDBC over SQL*Net
No encryptionXML according to canonical model
D
E
EJB over RMI
B
XML/SOAPHTTP
WS-Security
40
B
ORCHESTRATION, POTENTIALLY LONG RUNNING, STATEFUL, ASYNCHRONOUS
AD
E
Process(BPM/BPEL)
41
B
IMPLEMENTATION OF SERVICES
A ESB
Q
PL/SQL
Mail Server
42
B
EXTREME DECOUPLING USING EVENTS
A
D
Event Handler E
F
subscription
B
43Se
rvic
e Bu
s
SOA AND MOBILE DATABASE
44
SOA AND MOBILE DATABASE
• Services that hide underlying complexity– Technology, formats, protocols, data structure of
back end systems• Services that handle two-way data
synchronization– Possibly conflicting changes on both ends– Violation of server side data constraints– Smart refresh of data on mobile device
• ESB that supports subscriptions and push to send alerts/notifications/updates to mobile devices
• ESB that exposes services in appropriate protocol and format (e.g. http/json)
• ESB that handles security in transport and message contents
45Se
rvic
e Bu
s
SOA AND MOBILE DATABASE
46
SOA AND MOBILE DATABASE
• ESB ensuring the right availability – possibly exceeding the service window of back-end systems
• ESB dealing with potentially large number of devices– Scalability, Throttling (DoS protection)
• (Simulated) Device-to-Device-Push• Upgrade of meta-data, apps and data-
structure• Bulk upload of Big Data from mobile device• Initialization/Update of
pre-populated data on device
47
SUMMARY
• Mobile Database is a necessity to ensure apps can run– With acceptable performance– At all (in off-line situations)
• Mobile Database is a local cache that contains a snapshot of the ‘back end data’ at a certain timestamp– Cache is not complete and is not current– Cache contains data collected and edited on device
• Mobile Database is part of distributed database system– Without real-time distributed transactions– Periodic synchronization between Mobile Database and
Server is required (to really complete the transaction)– Data conflicts need to be resolved
• SOA can help to hide complexity of back end and offer available services with fitting protocol and data format– Providing security, handling peak loads, supporting push,
understanding delta-based updates, …
48
RESOURCES
• Blog: technology.amis.nl– On Oracle, SQL, Java, SOA, BPM & more
• Email: [email protected]
• : lucasjellema
• Slideshare: www.slideshare.net/lucasjellema
• : www.amis.nl, [email protected] +31 306016000 Edisonbaan 15, Nieuwegein
The Netherlands