Map Analytics in Starcraft II (2/3/2015)
-
Upload
gy8 -
Category
Data & Analytics
-
view
33 -
download
0
Transcript of Map Analytics in Starcraft II (2/3/2015)
![Page 1: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/1.jpg)
StarGazer: !
Map Analytics in StarCraft II
Guang Yang
![Page 2: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/2.jpg)
![Page 3: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/3.jpg)
![Page 4: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/4.jpg)
![Page 5: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/5.jpg)
![Page 6: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/6.jpg)
![Page 7: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/7.jpg)
![Page 8: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/8.jpg)
![Page 9: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/9.jpg)
(Demo)
![Page 10: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/10.jpg)
Pipeline
10
IngestionStreaming
Batch
Data-store
API/Front-end
![Page 11: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/11.jpg)
Ingestion
11
API
AWS EC2 Node
AWS EC2 Node
AWS EC2 Node
AWS EC2 Node
Serialize via Avro
Kafka Topics: !- Stream !- Batch
Example API Request: http://api.ggtracker.com/api/v1/matches/3529593.json
![Page 12: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/12.jpg)
Serialization via Avro
- reinforces schema !
- splittable on HDFS !
- backward compatible !- saves space (binary)
![Page 13: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/13.jpg)
Batch
13
Avro Files (on HDFS) SchemaRDD (in Spark SQL) Tables in Cassandra
- in-memory !- expressive commands
!- full benefits of RDD
- AP !- tunable consistency
![Page 14: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/14.jpg)
Streaming
14
![Page 15: Map Analytics in Starcraft II (2/3/2015)](https://reader030.fdocuments.us/reader030/viewer/2022032420/55a5f7bd1a28ab3e188b472e/html5/thumbnails/15.jpg)
Guang Yang
- B.A. in Computational and Applied Mathematics (Rice University)
- M.S. in Industrial Engineering & Operations Research (UC Berkeley)
- Got into Diamond League as Terran without making any Tanks
- Email: [email protected]
- GitHub: github.com/gy8
15
Shetland, Scotland, UK