Distributed systems in practice, in theory (ScaleConf Colombia)
-
Upload
aysylu-greenberg -
Category
Software
-
view
198 -
download
1
Transcript of Distributed systems in practice, in theory (ScaleConf Colombia)
![Page 1: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/1.jpg)
Aysylu GreenbergMarch 25, 2017
Distributed Systems in Practice,in Theory
![Page 2: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/2.jpg)
* 2
Aysylu Greenberg
![Page 3: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/3.jpg)
* 3
Aysylu Greenberg
paperswelove.org
![Page 4: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/4.jpg)
* 4
Aysylu Greenberg
@aysylu22
paperswelove.org
![Page 5: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/5.jpg)
How I got into reading papers as a
practitioner in industry
![Page 6: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/6.jpg)
Computer Science ResearchInDistributed Systems Industry
![Page 7: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/7.jpg)
Operating systems research
![Page 8: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/8.jpg)
Operating systems research
![Page 9: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/9.jpg)
Operating systems research
Concurrency
![Page 10: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/10.jpg)
Operating systems research
Concurrency
Concurrency primitives: mutex & semaphore
![Page 11: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/11.jpg)
Operating systems research
Concurrency
Concurrency primitives: mutex & semaphore
Processes execute at different speeds
![Page 12: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/12.jpg)
Time in distributed systems
https://www.flickr.com/photos/national_archives_of_norway/62633532281970
![Page 13: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/13.jpg)
Time in distributed systems
1970https://www.flickr.com/photos/national_archives_of_norway/6263353228
![Page 14: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/14.jpg)
https://www.flickr.com/photos/national_archives_of_norway/6263353228
Time in distributed systems
Pipelining
1970
![Page 15: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/15.jpg)
1980https://www.flickr.com/photos/findyoursearch/6848657291
![Page 16: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/16.jpg)
Distributed consensus
1980https://www.flickr.com/photos/findyoursearch/6848657291
![Page 17: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/17.jpg)
Distributed consensus
1980https://www.flickr.com/photos/findyoursearch/6848657291
![Page 18: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/18.jpg)
Distributed consensus
1980https://www.flickr.com/photos/findyoursearch/6848657291
![Page 19: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/19.jpg)
Distributed consensus
1980 Paxoshttps://www.flickr.com/photos/findyoursearch/6848657291
![Page 20: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/20.jpg)
Reconsider large systems
![Page 21: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/21.jpg)
Reconsider large systems
Shared infrastructure
...
![Page 22: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/22.jpg)
CS Research is Timeless
Inform decisions
Mitigate technical risk
![Page 23: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/23.jpg)
Today
● Staged Event-Driven Architecture
![Page 24: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/24.jpg)
Today
● Staged Event-Driven Architecture● Leases
![Page 25: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/25.jpg)
Today
● Staged Event-Driven Architecture● Leases● Inaccurate Computations
![Page 26: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/26.jpg)
Today
● Staged Event-Driven Architecture● Leases● Inaccurate Computations
![Page 27: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/27.jpg)
Staged Event Driven
Architecture&
Deep Pipelines
2001
![Page 28: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/28.jpg)
Hardware to Data Pipelines
![Page 29: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/29.jpg)
Hardware to Data Pipelines
https://en.wikipedia.org/wiki/Graphics_pipeline
![Page 30: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/30.jpg)
![Page 31: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/31.jpg)
Staged Event Driven Architecture
![Page 32: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/32.jpg)
Staged Event Driven Architecture
+ -
![Page 33: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/33.jpg)
Single-machine pipeline
generalizes to distributed pipelines
Staged Event Driven Architecture
![Page 34: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/34.jpg)
Search Indexing Pipelines
![Page 35: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/35.jpg)
Search Indexing Pipelines
![Page 36: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/36.jpg)
Search Indexing Pipelines
![Page 37: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/37.jpg)
Search Indexing Pipelines
![Page 38: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/38.jpg)
Search Indexing Pipelines
![Page 39: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/39.jpg)
Search Indexing Pipelines
![Page 40: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/40.jpg)
Search Indexing Pipelines
![Page 41: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/41.jpg)
Search Indexing Pipelines
+ -
![Page 42: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/42.jpg)
Today
● Staged Event-Driven Architecture● Leases● Inaccurate Computations
![Page 43: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/43.jpg)
Leasesas Heart Beat in
Distributed Systems
1989
![Page 44: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/44.jpg)
![Page 45: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/45.jpg)
Leases
● Distributed locking
![Page 46: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/46.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short
![Page 47: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/47.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long
![Page 48: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/48.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long● Use of leases in modern applications
○ Leader election
![Page 49: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/49.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long● Use of leases in modern applications
○ Leader election○ Liveness detection
![Page 50: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/50.jpg)
Build my sculpture
BoteroBuild
System
![Page 51: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/51.jpg)
Build my sculpture
BoteroBuild
System
OK
![Page 52: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/52.jpg)
Build my sculpture
BoteroBuild
System
OK
Waiting for the results
![Page 53: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/53.jpg)
Build my sculpture
BoteroBuild
System
OK
Waiting for the results
Build is in progress
![Page 54: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/54.jpg)
Build my sculpture
BoteroBuild
System
OK
Waiting for the results
Build is in progress
Waiting for the results
![Page 55: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/55.jpg)
Build my sculpture
BoteroBuild
System
OK
Waiting for the results
Build is in progress
Waiting for the results
Build is finished
![Page 56: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/56.jpg)
![Page 57: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/57.jpg)
Leases in Build System
![Page 58: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/58.jpg)
Leases in Build System
![Page 59: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/59.jpg)
Leases in Build System
![Page 60: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/60.jpg)
Leases in Build System
![Page 61: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/61.jpg)
Leases in Build System
![Page 62: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/62.jpg)
Leases in Build System
![Page 63: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/63.jpg)
Leases for heartbeat:How long should the lease term be?
![Page 64: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/64.jpg)
Today
● Staged Event-Driven Architecture● Leases● Inaccurate Computations
![Page 65: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/65.jpg)
Inaccurate Computations&Serving Search Results
![Page 66: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/66.jpg)
From Accurate to "Good Enough"
![Page 67: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/67.jpg)
[Trade off] Inaccuracy for Performance
![Page 68: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/68.jpg)
![Page 69: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/69.jpg)
![Page 70: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/70.jpg)
![Page 71: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/71.jpg)
[In production]Inaccuracy for Performance & Resilience
![Page 72: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/72.jpg)
Jeff Dean "Building Software Systems at Google and Lessons Learned", Stanford, 2010
![Page 73: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/73.jpg)
![Page 74: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/74.jpg)
![Page 75: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/75.jpg)
[Designing with]Inaccuracy for Performance & Resilience
![Page 76: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/76.jpg)
[Designing with]Inaccuracy for Performance & Resilience
simplified implementation
focus on observabilityapplicable to some problem domains
![Page 77: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/77.jpg)
[Designing with]Inaccuracy for Performance & Resilience
fuzz testing
generative testing
simplified implementation
fault injection testing
focus on observabilityapplicable to some problem domains
![Page 78: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/78.jpg)
References● F. Corbato, M. Daggett, R. Daley "An Experimental
Time-Sharing System"● E. Dijkstra "Cooperating Sequential Processes"● L. Lamport "Time, Clocks, and the Ordering of Events in a
Distributed System"● B. Oki, B. Liskov "Viewstamped Replication: A New Primary
Copy Method to Support Highly-Available Distributed Systems"● L. Lamport "The Part-Time Parliament"
![Page 79: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/79.jpg)
References● M. Welsh, D. Culler, E. Brewer "SEDA: An Architecture for
Well-Conditioned, Scalable Internet Services"● C. Gray, D. Cheriton "Leases: An Efficient Fault-Tolerant
Mechanism for Distributed File Cache Consistency"● S. Agarwal, B. Mozafari et al. "BlinkDB: Queries with Bounded
Errors and Bounded Response Times on Very Large Data"
![Page 80: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/80.jpg)
Robust & scalable pipelines
![Page 81: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/81.jpg)
Robust & scalable pipelinesLeases for sharing &
heartbeat
![Page 82: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/82.jpg)
Robust & scalable pipelinesLeases for sharing &
heartbeatInaccuracy for resilience &
performance
![Page 83: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/83.jpg)
Robust & scalable pipelinesLeases for sharing &
heartbeatInaccuracy for resilience &
performance
CS research is timeless:use it to mitigate risk
![Page 84: Distributed systems in practice, in theory (ScaleConf Colombia)](https://reader031.fdocuments.us/reader031/viewer/2022030313/58ec99761a28ab2f3b8b467d/html5/thumbnails/84.jpg)
Aysylu GreenbergMarch 25, 2017@aysylu22
Distributed Systems in Practice,in Theory