Web Development To Big Data: A Journey
@samelamin
@samelamin
Sam I Am
@samelamin
Why I Changed Careers?
WTF Is Big Data
@samelamin
The 3 Vs
@samelamin
@samelamin
Redshift Datawarehouse
@samelamin
State Of Play
@samelamin
Immutable Infrastructure
@samelamin
Slow Updates
@samelamin
Run Every 15 Mins
@samelamin
New Sources = Pain
@samelamin
Building A Data Pipelines
@samelamin
Make Data Queryable!
@samelamin
Architecture!
@samelamin
Handle Bad Data
@samelamin
Schema Updates!
@samelamin
Alerting!
@samelamin
Data Formats
@samelamin
Columnar Storage
@samelamin
Data Lake
@samelamin
Apache Spark
@samelamin
DAG Ting!
@samelamin
Data Serving Layer
@samelamin
Lambda Architecture
Data Streaming
@samelamin
@samelamin
Issues With Streaming
@samelamin
Structured Streaming AKA V2
@samelamin
Google BigQuery
@samelamin
Brag Alert!
https://github.com/samelamin/spark-bigquery
@samelamin
ML Powering Recommendations
@samelamin
Thank You!
@samelamin
Write to BQ DEMO!
@samelamin
Read From BQ DEMO!
@samelamin
Structured Streaming DEMO!