7/20/17

1

CH7:DatabaseswithAWS

By:DanPeekstok

Overview

•  QuickDatabaseoverview•  AmazonRelaFonalDatabaseService(RDS)

•  AmazonRedshiJ

•  AmazonDynamoDB

DatabasePrimer–RelaFonalDatabases

• Mostcommondatabasesystem

• Musthaveaprimarykeyandpredefineddatatypes

•  DefinedaseitherOnlineTransacFonProcessing(OLTP)orOnlineAnalyFcalProcessing(OLAP)•  OLTPisusedforfrequentread/writessuchasanapplicaFondatabase•  OLAPisdesignedfordatawarehousesoranalyzinglargedatasets

•  AmazonRDShandlesbotheasily

7/20/17

2

DatabasePrimer–DataWarehouses

•  Datacancomefrommanysources

•  UsuallyaspecifictypeofarelaFonaldatabasethatusesOLAPforanalysisandreporFng

•  AmazonRedshiJisahigh-performancedatawarehousedesignedspecificallyforOLAPusecases

•  CanalsobecombinedwithatradiFonalAmazonRDSdatabasefortransacFons

DatabasePrimer–NoSQL

•  HavegoWenmorepopularrecentlyduetoflexibilityandperformance

•  EasiertoscalebeyondasingleserverthantradiFonalrelaFonaldatabases•  AmazonDynamoDBisusefultohelpcreateandscaleaNoSQLdatabase

AmazonRDS–OverviewofFeatures

•  ThemainideaofRDSistoletthesetupandmaintenanceofthedatabasebeeasysoyoucanfocusontheapplicaFoninstead

•  RDSinstancesaren’tcreatedlikeEC2instances.Youdon’tgetshellaccesstotheinstance,justtheSQLendpoint.

•  However,youcanusetheAWSCLItomanagealotoftheservicesfromyourowncomputer,suchascreaFng,modifying,ordroppingadatabase

7/20/17

3

AmazonRDS–InstanceSizes

•  Databaseinstancesizes•  Smallestisadb.t2.microinstancewith1virtualCPUand1GBofRAM

•  Largestisadb.r3.8xlargeinstancewith32vCPUsand244GBofRAM

•  Databasememory•  Cangetdatabasesrangingfrom5GBupto64TBofstorage

•  YoucanchangethesizeofyourinstanceoverFmeshouldyourneedschange.ThismakesscalingasmallprogramintoalargeproducFoneasy•  Somedatabaseenginesevenallowforscalingacrossmanyserverssoeachcanrunaninstanceofthedatabasethatupdatestherest

AmazonRDS

•  Comparisonofwhoisresponsiblefordifferentaspectsofthedatabasewithdifferentservices

AmazonRDS–DatabaseEngines

•  SupportedDatabaseEngines•  Mysql

•  PostgreSQL•  MariaDB

•  Oracle•  MicrosoJSQLServer

•  AmazonAurora

•  ForOracleandMicrosoJSQLServer,youhavetochosealicenseopFon•  LicenseIncludedcostsmorebutAmazondealswiththelicenseforyou

•  BringYourOwnLicense(BYOL)meansyouhavetopurchaseandmanagethelicense

7/20/17

4

AmazonRDS–AmazonAurora

•  AuroraisAmazon’sowndatabasesystemthatoffersthesimplicityofanopensourcedatabaseandthetechnologyofanenterprise-gradedatabase

•  BuiltoffofMySQLwithamoreserviceorientedapproach

•  Becauseofthat,itscompaFblewithanyMySQLinterface

•  Duetothebehindthesceneschanges,itcandeliverupto5FmestheperformanceofMySQLwithoutanychangesfromtheuserorapplicaFon

• WhenanAurorainstanceiscreated,youcreateaclusterinstead.ThisallowsforacopyofthedatabaseoneachofmanydifferentAvailabilityZones

AmazonRDS–OpFons

•  StorageOptions•  MagneticDisk:cheap,poorestperformance,andsmallersizes

•  GeneralPurposeSSD:moderatecost,betterperformance,andlargersizes

•  ProvisionedIOPS:expensive,bestperformance,samesizesasgeneralpurposessds

•  Formostapplications,generalpurposessdsarethebestchoice.Theyoffergoodperformanceandlowercosts

AmazonRDS–BackupandRecovery

• MostprojectshaveeitheraRecoveryPointObjecFveorRecoveryTimeObjecFve.Thesespecifywhatisacceptablefordataloss

•  AmazonoffersAutomatedBackupsforyou.ThesewillbackuptheenFreDBinstance,notjustthedata.YoucanchangehowoJenitbacksuptheinstance

•  InaddiFon,AmazonoffersManualDBSnapshotsthatcanbedoneatanyFmethroughtheAWSCLI.

•  RestoringfromeitherasnapshotorabackupisveryeasythroughtheAWSconsole

7/20/17

5

AmazonRDS–UsingMulti-AZ

• Multi-AZisawaytohelpwithdatarecoverybycopyingyourdatabaseacrossotheravailabilityzonestohelpmeetRPOandRTOtargets

•  Youwillhaveonemasterdatabaseandanumberofslaves.Yourapplicationwillupdatethemasterwhichwillthenupdatetheslaves

•  Sincethedatabaseisclonesacrossavailabilityzones,ifonegoesdown,youstillhaveaccesstotheothersforrecovery

AmazonRedshiJ

•  Petabytescaledatawarehouseservice•  RelaFonaldatabasedesignedforOLAPscenarioswithafocusonperformance

•  BasedonPostgreSQL,soexisFngSQLclientscaneasilyuseitwithafewsmallchanges

•  CancreateSnapshotsjustlikewithAmazonRDS

•  UsesIAMusersforprimarysecuritywithnetworksecuritytopreventIPaddressesfromconnecFng

AmazonRedshiJ–

ClustersandNodes

•  ApplicaFonsonlyinteractwiththeleadernode.Thattalkswiththeothernodes

•  6typesofnodesareavailablein2categories:•  DenseCompute

•  DenseStorage

7/20/17

6

AmazonRedshift–Tables

•  UsesstandardtablecreaFonstatementslikeCREATE TABLEandalsoaddsopFonssuchascompressionencoding,distribuFonstrategy,andsortkeys

•  Compressionencodingishowtocompressthedata.Itcandoitonitsownoryoucanspecifyhow

•  DistribuFonStrategyishowtodistributethedataacrossthenodes•  SortKeysiswhatcolumnscanbesortedmakingsearchandsorFngonthatkeyfaster

AmazonRedshiJ–Data

•  RedshiJaddsanewcommandCOPYthatisfasterforloadingdataintoatable

•  Builttoloaddatainfrombulkfilesandcanreadmanyfilesatonce

•  AJerloadingdata,runVACUUMtoreorganizethedataandreclaimspacefromdeletes

•  SupportsstandardSELECTstatementsforqueries

•  CanuseCloudWatchtoanalyzeperformance

AmazonDynamoDB

•  ProvidesaNoSQLdatabasewithfast,lowlatencyperformancethatiseasytoscaleverFcallyandhorizontally•  SinceitismanagedbyAmazon,youdon’thavetoworryaboutscalingit,justusingit•  EachitemhasaprimarykeyandoneormoreaWributes•  Eachitemisakey/valuepair

•  NodatatypesontheaWributes•  Maximumof400KBperitem

•  ConnectviaHTTP/SendpointandprovideJSONdataforread/writes

7/20/17

7

AmazonDynamoDB–PrimarykeysandDataTypes

•  Eachprimarykeymusthaveadatatype

•  PrimaryKeysareoneof2opFons•  PrimaryKeyisjusttheonefieldthatishashed

•  PrimaryKeyandSortKeydefinestwoitemsthatareusedlikeadoubleprimarykey.ItemsmusthaveauniquecombinaFonofthetwovalues

•  Candefinesecondaryindexesthatcanbeusedforlookupsoneachtable

AmazonDynamoDB–ReadingandWriFngData

•  MostusersuseAmazon’sprovidedSDKsforusewithapplicaFons•  4mainoperaFons•  PutItemcreatesanewitemorupdatesthatitemifitexists

•  UpdateItemchangestheaWributesofanexisFngitem

•  DeleteItemremovesanitemfromthetable

•  GetItemretrievesanitembasedontheprovidedprimarykey

•  DynamoDBisreferredtoaseventuallyconsistentmeaningthatareadjustaJerawritemaynotshowthedata•  ThisisduetohowthedataisspreadacrossmulFpleservers

•  AStronglyConsistentopFonisavailablethatwillforceupdatesbeforethenextquerybutitcostssomeefficiency

AmazonDynamaDB–ScalingandParFFoning

•  Eachtable’sitemsarestoredonmulFpleparFFonsforfasterreads

• WhichparFFonthedataisstoredonisbasedonthehashedprimarykey

•  EachparFFoncanholdupto10GBofdataanditautomaFcallymakesmoreparFFonsbasedonneedandsplitthecurrentdata

•  EachparFFonhasroomforusagespikessoifthedataisneededfasteratonepoint,itcandothat

7/20/17

8

AmazonDynamoDB–DynamoDBStreams

• ManyNoSQLapplicaFonsneedtotracknewitemsorchangesforaddiFonaldataprocessing

•  DynamoDBprovidesaccesstoaDynamoDBStreamthatwillshowalistofallthechangestoitemsinthelast24hours.Thisprovideseasyaccesstoalistofrecentchangesforprocessing

•  Streamscanbeenables,disabled,andaccessedfromtheAWSConsole,CLI,andSDKs