Post on 06-Oct-2020
Streaming Data Challenges
The Data Dossier
Choose a Lesson
Cloud Pub/Sub Overview
Return to Table of Contents
Pub/Sub Hands On
Text
Connecting Kafka to GCP
Monitoring Subscriber Health
What is Streaming Data?- "Unbounded" data- Infinite, never completes, always flowing
Examples
Fast action is often necessary- Quickly collect data, gain insights, and take action- Sending to storage can add latency- Use cases:
- Credit card fraud detection- Predicting highway traffic
Next
Streaming Data Challenges
Traffic Sensors Credit Card Transactions Mobile Gaming
Streaming Data Challenges
The Data Dossier
Choose a Lesson
Cloud Pub/Sub Overview
Return to Table of Contents
Pub/Sub Hands On
Text
Connecting Kafka to GCP
Monitoring Subscriber Health
Tight vs. Loose Coupling in Systems- Tightly (direct) coupled systems more likely to fail- Loosely coupled systems with 'buffer' scale have better fault
tolerance
Previous
Streaming Data Challenges
Sender Sender
Receiver
Sender Sender
Overloaded Receiver
Lost messages, delays in processing
Tightly-Coupled System
Publisher
Subscriber
Publisher
Buffer/Message Bus
Publisher
Subscriber
Buffer/Message Bus
Subscriber
Loosely-Coupled System- Fault tolerance- Scalability- Message queuing