Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
-
Upload
gigaom -
Category
Technology
-
view
107 -
download
2
description
Transcript of Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
![Page 1: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/1.jpg)
1
![Page 2: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/2.jpg)
Last Updated: Jan. 2014
VP Pla&orm Evangelism @cobiacomm on Twi6er
h6p://blog.cobia.net/cobiacomm/
Chris Haddad
Tracking a Soccer Game with Big Data
![Page 3: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/3.jpg)
Big Data Improves Team Performance
๏ Sports Teams
๏ Your Business Team 3
![Page 4: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/4.jpg)
Improvement Requires A Feedback Loop
4
![Page 5: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/5.jpg)
Internet of Things (IoT) Bridges Physical and Virtual Worlds
๏ Sensors, actuators everywhere
๏ Internet connecOvity
Improvement = IoT, Apps, and APIs
๏ VisualizaOon Apps
๏ Control APIs
![Page 6: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/6.jpg)
DEBS Challenge
• Bridge game play to Internet of Things
• Sensors in shoes, ball, goalie hands
• Analyze Soccer Game
• SpaOal and Temporal Processing
• Visualize Game Play
• AcOvity Heat maps
• Recommend Performance Improvements
http://srinathsview.blogspot.com/2013/05/solving-debs-2013-grand-challenge-with.html
![Page 7: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/7.jpg)
Bridging Game Play to IoT
7
Temporal Tracking
SpaOal Tracking
• Each event includes the locaOon (x,y,z), Ome stamp, velocity and acceleraOon
• Throughput 15,000 events per second
![Page 8: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/8.jpg)
Analyze Game
8
๏ BALL POSSESSION STATE DIAGRAM
๏ SHOT ON GOAL STATE DIAGRAM
![Page 9: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/9.jpg)
Analyze Game
9
๏ RUNNING LOCATION STATES
![Page 10: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/10.jpg)
10
SoluOon Architecture ๏ Capture millions of events with
Common Events Collector
๏ Derive real-‐Ome decisions with temporal processing
๏ Further opOmize recommendaOons with spaOal Map-‐Reduce processing analyOcs
๏ View feedback dashboards
![Page 11: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/11.jpg)
Batch AnalyOc FoundaOon
11
![Page 12: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/12.jpg)
Business AcOvity Monitor
![Page 13: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/13.jpg)
Realizing Real-‐Ome AnalyOcs ๏ Processing Data on the fly, while
storing a minimal amount of informaOon and responding fast (from <1 ms to few seconds)
๏ Idea of Event streams
๏ A series of events in Ome
๏ Enabling technologies
๏ Stream Processing (Storm)
๏ Complex Event processing (Siddhi)
![Page 14: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/14.jpg)
Complex Event Processing
![Page 15: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/15.jpg)
![Page 16: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/16.jpg)
Complex Event Processing
Stream Processing
• SQL like language • Supports powerful temporal operators (e.g. windows, event paberns) • Focus on speed • Harder to scale • e.g. WSO2 CEP, Streambase, Esper
• Operators connected in a network, but you have to write the logic • Distributed by design • Focus on reliability (do not loose messages), has transacOons • e.g. Storm, S4
![Page 17: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/17.jpg)
Complex Event Processing Operators ๏ Filters or transformaOons (process a single event)
๏ from Ball[v>10] select .. insert into ..
๏ Windows + aggregaOon (track window of events: Ome, length)
๏ from Ball#window.time(30s) select avg(v) ..
๏ Joins (join two event streams to one)
๏ from Ball#window.time(30s) as b join Players as p on p.v < b.v
๏ Paberns (state machine implementaOon)
๏ from Ball[v>10], Ball[v<10]*,Ball[v>10] select ..
๏ Event tables (map a database as an event stream)
๏ Define table HitV (v double) using .. db info ..
![Page 18: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/18.jpg)
Usecase 1: Running Analysis ๏ Detect when speed crosses threshold limits
define partition player by Players .id; from s = Players [v <= 1 or v > 11] , t = Players [v > 1 and v <= 11]+ , e = Players [v <= 1 or v > 11] select s.ts as tsStart , e.ts as tsStop ,s.id as playerId , ‘‘trot" as intensity , t [0].v as instantSpeed , (e.ts - s.ts )/1000000000 as unitPeriod insert into RunningStats partition by player;
![Page 19: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/19.jpg)
Usecase 2: Ball possession ๏ Ball possession
๏ Defined as possessing the ball from Ome you hit it unOl someone else hit it or ball leaves the ground)
![Page 20: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/20.jpg)
Usecase 3: Heatmap of AcOvity
๏ Show where acOons happened (via cells defined by a grid of 64X100 etc.), need updates once every second
๏ Can solved via cell change boundaries, but does not work if one player stays more than 1 sec in the same cell. So need to join with a Omer.
![Page 21: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/21.jpg)
Usecase 4: Detect kicks on goal
๏ Main Idea: Detect kicks on the ball, calculate direcOon aker 1m, and keep giving updates as long as it is in right direcOon
![Page 22: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/22.jpg)
Results for DEBS Scenarios
![Page 23: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/23.jpg)
Big Data and IoT Changes the Game ๏ Real-‐Ome Status Dashboard
๏ Player game play
๏ Momentum indicators
๏ PredicOons about the next move
๏ Improvement Dashboard
๏ Study of game and players effecOveness
๏ Monitor player health and body funcOons
![Page 24: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/24.jpg)
Success Target
Photo by John Trainoron Flickr http://www.flickr.com/photos/trainor/2902023575/, Licensed under CC
![Page 25: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/25.jpg)
SituaOonal Context: Traffic Example
{incidents: [ {impacting: false,
eventCode: 0, iconURL: "http://...",
lat: 38.743999, shortDesc: "Exit ramp closed on I 95...",
type: 1, severity: 0,
fullDesc: "In NEWINGTON exit ramp closed on I-95 ...",
startTime: "2010-02-21T00:14:07", lng: -77.188004, id: "368598263",
endTime: "2010-02-27T05:04:19" },
Image source: http://www.directoryofnewyorkcity.com/blog/2009/05/how-to-find-parking-in-new-york-city/ Real-time traffic map: http://www.mapquestapi.com/traffic/
![Page 26: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/26.jpg)
Accelerate interac6ons inside and outside the organiza6on
Reduce interac6on
fric6on and cost
Increase engagement and
enhance produc6vity
Sense business ac6vity
and automa6cally
adapt
Become a More Connected Business
http://wso2.com/landing/enabling-the-connected-business
![Page 27: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/27.jpg)
Connected Business Reference Architecture
![Page 28: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/28.jpg)
Conceptual Architecture
![Page 29: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/29.jpg)
CEP High Availability
![Page 30: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/30.jpg)
Siddhi Storm Bolt (next Major Release)
๏ We have wriben a Siddhi bolt that would let users run distributed Siddhi Queries using Storm
SiddhiBolt siddhiBolt1 = new SiddhiBolt( .. siddhi queries .. ); SiddhiBolt siddhiBolt2 = new SiddhiBolt( .. siddhi queries .. ); TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("source", new PlayStream(), 1); builder.setBolt("node1", siddhiBolt1, 1) .shuffleGrouping("source", "PlayStream1"); .. builder.setBolt("LeafEacho", new EchoBolt(), 1) .shuffleGrouping("node1", "LongAdvanceStream"); .. cluster.submitTopology("word-count", conf, builder.createTopology());
![Page 31: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/31.jpg)
31
![Page 32: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/32.jpg)
Contact us !
![Page 33: Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad](https://reader033.fdocuments.us/reader033/viewer/2022051412/54c7a5e74a7959e7478b465b/html5/thumbnails/33.jpg)
33
About the Presenter
๏ Chris Haddad
๏ VP Plaoorm Evangelism
๏ Learn more about me
๏ www.linkedin.com/in/cobiacomm/
๏ Follow me
๏ @cobiacomm on Twiber
๏ hbp://blog.cobia.net/cobiacomm
๏ On Google+ too