Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different...
Transcript of Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different...
![Page 1: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/1.jpg)
Joining Punctuated Streams.Luping Ding, Nishant Mehta, Elke A. Rundensteiner, and George T.
1
![Page 2: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/2.jpg)
Motivation.• Traditional Join: Symmetric Hash, etc, but…• Join on Streams
Steam is potentially unbounded : Memory overflow
2
![Page 3: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/3.jpg)
Existing Approach.•Memory & Disk: XJoin•Sliding Window
Paper’s Approach.•Punctuation: PJoin
3
![Page 4: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/4.jpg)
• What is Punctuation?“ordered set of patterns”“signal end of transmitting certain attribute values”
4
![Page 5: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/5.jpg)
•How PJoin use punctuation?
•Why use punctuation?
•How does PJoin work?
Short answer: Purge & Propagation
5
![Page 6: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/6.jpg)
6
PJoin, in general.★ When is good to purge to tuple? ★ When is good to propagate a punctuation?
![Page 7: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/7.jpg)
★ When is good to purge to tuple?First, let’s introduce some denotation …
When is it safe to purge a tuple from a stream?
Purge Mode: Eager Purge !! ! ! ! ! Lazy Purge (with a purge threshold)
7
![Page 8: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/8.jpg)
★ When is good to propagate a punctuation?
When is it safe to propagate a punctuation?
![Page 9: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/9.jpg)
A propagation optimizer: Punctuation Index
• Eager & Lazy build • Pull & Pull propagation
![Page 10: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/10.jpg)
For Experimenting PJoin: An Event Driven Framework
Monitor runtime parameters
and join state
Listeners:
Invoke Events:
1. “keep track of a variety of runtime parameters” 2. “model the different coupling alternatives among components”
10
![Page 11: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/11.jpg)
Experiments Setup.•Raindrop
•Synthetic stream
•Data with poisson inter-arrival time
•Many-to-many relationship
![Page 12: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/12.jpg)
Are we able to reduce memory overhead? (if so, how much?)
!
Are we able to increase data throughput? (if so, how much?)
![Page 13: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/13.jpg)
Round-1: PJoin VS. XJoinMemory Overhead:
![Page 14: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/14.jpg)
Round-1: PJoin VS. XJoinData Throughput:
![Page 15: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/15.jpg)
Round-2: Eager purge VS. Lazy purgeMemory Overhead:
![Page 16: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/16.jpg)
Round-2: Eager purge VS. Lazy purgeData Throughput:
![Page 17: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/17.jpg)
A closer look at PJoin - 1: Different punctuation inter-arrival time
![Page 18: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/18.jpg)
A closer look at PJoin - 2: Asymmetric punctuation inter-arrival time
Memory Overhead:
![Page 19: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/19.jpg)
A closer look at PJoin - 2: Asymmetric punctuation inter-arrival time
Data Throughput:
![Page 20: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/20.jpg)
A closer look at PJoin - 2.5: Asymmetric punctuation inter-arrival time
Combined with Different Purge ModesMemory Overhead:
![Page 21: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/21.jpg)
A closer look at PJoin - 2.5: Asymmetric punctuation inter-arrival time
Combined with Different Purge ModesData Throughput
![Page 22: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/22.jpg)
A closer look at PJoin - 3: Punctuation Propagation Capability
![Page 23: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/23.jpg)
Future Work
•To support sliding window
•To support n-ary join
![Page 24: Joining Punctuated Streams. · Asymmetric punctuation inter-arrival time Combined with Different Purge Modes Memory Overhead: A closer look at PJoin - 2.5: Asymmetric punctuation](https://reader034.fdocuments.us/reader034/viewer/2022050406/5f83e225f9fc7a44392b9979/html5/thumbnails/24.jpg)
Reference
Joining Punctuated Streams, Luping Ding, Nishant Mehta, Elke A. Rundensteiner, and George T.