DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale...

15
2 L22: NoSQL CS3200 Database design (sp18 s2) https://course.ccs.neu.edu/cs3200sp18s2/ 4/5/2018 Several slides courtesy of Benny Kimelfeld

Transcript of DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale...

Page 1: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

2

L22:NoSQL

CS3200 Databasedesign(sp18 s2)https://course.ccs.neu.edu/cs3200sp18s2/4/5/2018 SeveralslidescourtesyofBennyKimelfeld

Page 2: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

3

Outline

• Introduction• TransactionConsistency• 4maindatamodels- DocumentStores(e.g.,MongoDB)- Key-ValueStores(e.g.,Redis)- GraphDatabases(e.g.,Neo4j)- Column-FamilyStores

• ConcludingRemarks

Page 3: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

4

SQLMeansMorethanSQL

• SQLstandsforthequerylanguage• ButcommonlyreferstothetraditionalRDBMS:- Relationalstorage ofdata

• Eachtupleisstoredconsecutively- Joins asfirst-classcitizens

• Infact,normalformspreferjoinstomaintenance- Strongguarantees ontransactionmanagement

• Noconsistencyworrieswhenmanytransactionsoperatesimultaneouslyoncommondata

• Focusonscalingup- Thatis,makeasinglemachinedomore,faster

Page 4: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

5

Verticalvs.HorizontalScaling

"scaleup"

• Verticalscaling("scaleup"):youscalebyaddingmorepower(CPU,RAM)

• Horizontalscaling("scaleout"): youscalebyaddingmoremachines

"scaleout"

Page 5: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

6

TrendsDriveCommonRequirements

Social media + mobile computing

• Explosion in data, always available, constantly read and updated

• High load of simple requests of a common nature

• Some consistency can be compromised (e.g., 👍 )

Cloud computing + open source

• Affordable resources for management / analysis of data

• People of various skills / budgets need software solutions for distributed analysis of massive data

Database solutions need to scale out(utilize distribution, “scale horizontally”)

Page 6: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

7

CompromisesRequired

Whatisneededforeffectivedistributed,data-anduser-intensiveapplications?

1. Usedatamodelsandstoragethatallowtoavoidjoinsofbigobjects

2. Relaxtheguaranteesonconsistency

Page 7: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

8

NoSQL

• NotOnlySQL- Nottheotherthing!- TermintroducedbyCarloStrozzi in1998todescribeanalternativedatabasemodel

- Becamethenameofamovement followingEricEvans’sreuseforadistributed-databaseevent

• Seminalpapers:- Google’sBigTable

• Chang,Dean,Ghemawat,Hsieh,Wallach,Burrows,Chandra,Fikes,Gruber:Bigtable:ADistributedStorageSystemforStructuredData.OSDI2006:205-218

- Amazon’sDynamoDB• DeCandia,Hastorun,Jampani,Kakulapati,Lakshman,Pilchin,Sivasubramanian,Vosshall,Vogels:Dynamo:amazon'shighly

availablekey-valuestore.SOSP2007:205-220

Page 8: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

9

NoSQL fromnosql-database.org

“• NextGenerationDatabasesmostlyaddressingsomeofthepoints:beingnon-relational,distributed,open-source andhorizontallyscalable.

• Theoriginalintentionhasbeenmodernweb-scaledatabases.Themovementbeganearly2009andisgrowingrapidly.Oftenmorecharacteristicsapplysuchas:schema-free,easyreplicationsupport,simpleAPI,eventuallyconsistent/BASE(notACID),ahugeamountofdataandmore.

• Sothemisleadingterm“nosql”(thecommunitynowtranslatesitmostlywith“notonlysql”)shouldbeseenasanaliastosomethinglikethedefinitionabove.

”

Page 9: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

10

WhatisNoSQL?

Source:GeekandPoke:http://geekandpoke.typepad.com/geekandpoke/2011/01/nosql.html

Page 10: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

11

CommonNoSQL Features

• Non-relationaldatamodels• Flexiblestructure- Noneedtofixaschema,attributescanbeaddedandreplacedonthefly

• Massiveread/writeperformance;availabilityviahorizontalscaling- Replication andsharding (datapartitioning)- Potentiallythousandsofmachinesworldwide

• Opensource(veryoften)• APIstoimposelocality

Page 11: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

12

Whenthedatabasegrows:PartitioningTables

Source:http://cloudgirl.tech/data-partitioning-vertical-horizontal-hybrid-partitioning/

Page 12: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

13Source:http://cloudgirl.tech/data-partitioning-vertical-horizontal-hybrid-partitioning/

Verticalvs.Horisontal Partitioning

Page 13: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

14

HorizontalPartitioning("sharding")

Source:http://cloudgirl.tech/data-partitioning-vertical-horizontal-hybrid-partitioning/

Page 14: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

15

Verticalvs.Horizontalpartitioning

Source:http://www.piyushgupta.co.uk/2016/04/database-scaling-jargons.html,http://slideplayer.com/slide/12131436/70/images/17/SQL+Azure+Azure+Custom+Sharding.jpg

Page 15: DB - NoSQL - v180405d · Vertical vs. Horizontal Scaling "scale up" • Vertical scaling ("scale up"): you scale by adding more power (CPU, RAM) • Horizontal scaling ("scale out"):you

16Source:Anefficientschemeforprobabilisticskylinequeriesoverdistributeduncertaindata,Xiaoyong Li,Yijie Wang,Jie Yu


Effects of changing scale on landscape pattern analysis ...leml.la.asu.edu/Scaling-Class/Reading_PDFs/Wu-2004-scaling...Effects of changing scale on landscape pattern analysis: scaling

Effects of changing scale on landscape pattern analysis ...leml.la.asu.edu/Scaling-Class/Reading_PDFs/Wu-2004-scaling...Effects of changing scale on landscape pattern analysis: scaling

3.0 Methods for Scaling Crown Timber...3.0 Methods for Scaling Crown Timber When scaling sample scale loads at mass scale sites, there are two primary approved scaling methods: the

3.0 Methods for Scaling Crown Timber...3.0 Methods for Scaling Crown Timber When scaling sample scale loads at mass scale sites, there are two primary approved scaling methods: the

Analysis of Horizontal Reactive Scaling on Container ...

Analysis of Horizontal Reactive Scaling on Container ...

Scaling without Scale is Possible!

Scaling without Scale is Possible!

Scale & Scaling What is scale? Why is scale important in landscape ecology?

Scale & Scaling What is scale? Why is scale important in landscape ecology?

Load Balancers for VNF chains Horizontal Scaling

Load Balancers for VNF chains Horizontal Scaling

Horizontal Boundaries from Scale and Scope Economies

Horizontal Boundaries from Scale and Scope Economies

Data Management in Large-Scale Distributed Systems ... · Architecture of a data center A shared-nothing architecture Horizontal scaling No speci c hardware A hierarchical infrastructure

Data Management in Large-Scale Distributed Systems ... · Architecture of a data center A shared-nothing architecture Horizontal scaling No speci c hardware A hierarchical infrastructure

Planning for the Horizontal: Scaling Node.js Applications

Planning for the Horizontal: Scaling Node.js Applications

Agroecology: Scaling-up, scaling-out - ActionAid USA€¦ · Agroecology: Scaling-up, scaling-out ActionAid joins growing global calls to ‘scale-up’ and ‘scale-out’ agroecology.

Agroecology: Scaling-up, scaling-out - ActionAid USA€¦ · Agroecology: Scaling-up, scaling-out ActionAid joins growing global calls to ‘scale-up’ and ‘scale-out’ agroecology.

EDB Postgres High Availability and Horizontal Read Scaling ......EDB Postgres High Availability and Horizontal Read Scaling Architecture, Release 3.10 1.1Failover Manager Overview

EDB Postgres High Availability and Horizontal Read Scaling ......EDB Postgres High Availability and Horizontal Read Scaling Architecture, Release 3.10 1.1Failover Manager Overview

Secrecy Capacity Scaling of Large-Scale Cognitive Networks

Secrecy Capacity Scaling of Large-Scale Cognitive Networks

Scaling Agile with LeSS (Large Scale Scrum)

Scaling Agile with LeSS (Large Scale Scrum)

Indexing in Large Scale Image Collections: Scaling - CiteSeerX

Indexing in Large Scale Image Collections: Scaling - CiteSeerX

Cloud Scalability Patterns · Scaling Up == Vertical Scaling Scaling Out == Horizontal Scaling •Architectural Decision –ig decision… hard to change. Scaling Up: Scaling the

Cloud Scalability Patterns · Scaling Up == Vertical Scaling Scaling Out == Horizontal Scaling •Architectural Decision –ig decision… hard to change. Scaling Up: Scaling the

MySQL Sharding: Tools and Best Practices for Horizontal Scaling

MySQL Sharding: Tools and Best Practices for Horizontal Scaling

New Rules: Scaling Performance for Extreme Scale Computing

New Rules: Scaling Performance for Extreme Scale Computing

LKCE14 Kanban at Scale, Scaling Kanban

LKCE14 Kanban at Scale, Scaling Kanban

Cloud Scale Load Balancing - Rutgers Universitybadri/552dir/notes/W13-LB-four.pdf · Scaling •Vertical scaling (scale up) vs Horizontal scaling •Vertical scaling –More resource/capacity

Cloud Scale Load Balancing - Rutgers Universitybadri/552dir/notes/W13-LB-four.pdf · Scaling •Vertical scaling (scale up) vs Horizontal scaling •Vertical scaling –More resource/capacity

Distributed Data Analytics Thorsten Papenbrock Foundations ...€¦ · Vertical scaling (scale up) Add CPUs, RAM, Disk Replace entire machine Horizontal scaling (scale out) Add additional

Distributed Data Analytics Thorsten Papenbrock Foundations ...€¦ · Vertical scaling (scale up) Add CPUs, RAM, Disk Replace entire machine Horizontal scaling (scale out) Add additional