Oracle Database 12c IT WARS - nloug.nl · PDF filecommitment to deliver any material, code ......

Copyright©2014,Oracleand/oritsaffiliates.Allrightsreserved.|

OracleDatabase12candDevOpsMariaColganOracleDatabaseSystemsJune,2017@SQLMaria

IT

WARS


ThefollowingisintendedtooutlineourgeneralproductdirecNon.ItisintendedforinformaNonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncNonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andNmingofanyfeaturesorfuncNonalitydescribedforOracle’sproductsremainsatthesolediscreNonofOracle.

SafeHarborStatement


• ReputaNon:Fast&Agile

• Primaryfocus:Delivery

•  Favoriteline:“Let’sgetitdone”

Public 3

MeettheDeveloper

Developer


• ReputaNon:Cau=ous&RiskAverse

• Primaryfocus:Longevity&Security

•  Favoriteline:“Let’snotgethacked”

Public 4

MeettheDBA

DBA

Copyright©2015,Oracleand/oritsaffiliates.Allrightsreserved.| OracleConfidenNal–Internal/Restricted/HighlyRestricted 5

Together they must build a new web application for the recently

acquired coffee shops chain

Copyright©2015,Oracleand/oritsaffiliates.Allrightsreserved.| 6Public

Weneedtobuildanewwebapplica=onforthecoffee

shopswerecentlyacquiredinahurry,sowewanttouseaflexibleschemausingJSONdocuments.CanyousetupaMongoDBdatabaseforus?

ShecannotdifferenNateablasterfroma

sNck…

Developer DBA

Ohno!Whatdoeshewant

now?

Whydon’tyoujustuseOracle?

Copyright©2015,Oracleand/oritsaffiliates.Allrightsreserved.| 7

OracleDatabase12casaDocumentStore12.2JSON

SQL> CREATE TABLE orders (order_details BLOB CONSTRAINT is_json CHECK (order_details IS JSON) );

TablecontainingJSONdocuments

Public

•  OraclestoresJSONintablecolumns•  Nospecialdatatype•  CanbeVARCHAR2,BLOBorCLOB

•  JSONsupportedbyallOraclefeatures•  AnalyNcs,EncrypNon,In-Memory,RAC,

ReplicaNon,ParallelSQL,…

•  PluscanindexanyJSONelement


OracleDatabase12casaDocumentStore12.2JSON

SQL> CREATE TABLE orders (order_details BLOB CONSTRAINT is_json CHECK (order_details IS JSON) );


Public

•  OraclestoresJSONintablecolumns•  Nospecialdatatype•  CanbeVARCHAR2,BLOBorCLOB

•  JSONsupportedbyallOraclefeatures•  AnalyNcs,EncrypNon,In-Memory,RAC,

ReplicaNon,ParallelSQL,…

•  PluscanindexanyJSONelementIS_JSONconstraintensuresonlylegalJSONdocumentscanbeinserted


ButtheJSONdocumentisBLOB.That’snotwhatweneed.Weneedtobe

abletoquerytheelementsintheJSON

documents?

Developer DBA

Don’tpanic!OracleoffersnaNveSQL

supportforJSON.Letmeshowyou


NaNveSQLSupportforJSON12.2JSON

Public

•  JSONcanbequeriedusingsimpleSQLdotnotaNon

SELECT

o.order_details.location.city FROM orders o;

Location--------------------BostonLos AngelesNew YorkSan FranciscoWashington

SQL> desc ORDERSNAME TYPE------------------ -----------ORDER_DETAILS BLOB


{"date": "2017-05-03 10:40:36","loyaltyCardNo": 1230, "location": { "city": "Dallas", "state": "TX”}, "salesAmount": 7.55, "order": [ { "item": "Espresso”, …. }}


Afewhourslater….........

Public 11


Developer

12Public

Hey,wehavedecidedtodoeverythingviaRESTandweknowthatMongoDBcanspeakRESTSoitlookslikeweneedadocumentstore

aVerall

DBA

Dude,RESTisoutoftheboxwithOracle!Letme

showyou

Copyright©2015,Oracleand/oritsaffiliates.Allrightsreserved.| OracleConfidenNal–Internal/Restricted/HighlyRestricted 13

Conn establishment

Ackn

Request

Response

Conn close

Ackn

13

Request

Response

Conventional communication

RESTful communication

RESTinanutshell


OracleRESTDataServicesHTTP(s)APIApp-DevwithRelaNonalTablesinOracleDatabase

{“loyaltyCardNo”:1001,“name”:“ScooKing”,“LocaNon”:“500Mainstreet,InnovaNonCA”“date”:“Feb27,2017”"salesAmount":4.15“orders”:[{"item":"Mocha","size":"medium","salesAmount":4.15,"coffee_used_gram":15}]}

OracleRESTDataServices

ORDSmapsstandardURIrequeststocorrespondingrelaNonalSQL(notschemaless):e.g.SQLSELECTfromcustomersandorderstable.ORDSalsotransformstheSQLresultsintothehighlypopularJavaScriptObjectNotaNon(JSON),otherformatsincludeHTML,binaryandCSV.

FullycommioedtosupporNnganyandallstandardsrequiredbyFusion/SaaS/FMW;weareacNvelyengagedintheongoingdialog.

SQL

URIRequestorHTTP(s)postmappedtoSQLrequest

JDBCConnecNon

Pool

JSONSQLresultto

JSONtransformaNon

hops://myhost/myapp/hr/customerorders/1001

URI


ConfiguringRESTwithOracleExadataExpressCloudService

15



16

Givetheserviceanameandspecify

theprefix



17

SpecifytherestoftheURL

orders



18

Specifythetypeofservice

Specifytheactualcommandtobe

usedbytheservice


RESTEndpointForDeveloper

hops://demo-a425370.db.us2.oraclecloudapps.com/apex/coffeeshop/orders

OracleConfidenNal–Internal/Restricted/HighlyRestricted 19


Developer DBA

Goahead,sendyourRESTcalls


CheckingresultofRESTSELECT Count(*) FROM orders;

COUNT(*)

--------

1

21


DBA

22Public

Thisisawesome!YoujustreducedourNmetomarketbytwoweeks!Thanksa

million!

Theforceisstrongwithme

You’rewelcome

Developer


Afewhourslater….........

Public 23


DBA

24Public

I’mheadinghomeforthedayandIwanttokeep

workingonmycommutebutIwon’thavegoodinternetconnecNvity.Canwebuildthesameenvironmentin

Docker?

Absolutely!YouwillneedtodownloadtheDocker

imageforOracleDatabase12c&Oracle

SQLDeveloper

Developer

Letmeshowyouonmymachine


1.  Clickonthetoolsmenu2.  ClickontheRESTDataServices3.  ClickonInstall


EnablingSQLDeveloperasORDS


1.  UseOracleRESTDataServicethat’sincludedwithSQLDev

2.  SpecifythelocaNonforyourRESTconfigfile

3.  Clicknext




1.  FillinyoutheServiceNameforyourdatabaseindocker

2.  SpecifythepasswordforyourORDS_PUBLIC_USER

3.  Clicknext




1.  ClickRunStandalonemode2.  SpecifyHTTPPORT(8080)3.  Clicknext4.  Clickfinish




RESTEndpointForDocker

hop://localhost:8080/ords/coffeeshop/orders/order



Afewdayslater….........

Public 30


DBA

31Public

How’sitgoing?

Isn’ttheCRMdataalreadyinthedatabasewiththeJSONdata?

Yeah,exactly.Nowwejustneedtobuildsomeclassesto

getthatcorrespondingrelaNonaldatafortheJSON

orders

Hey,weareworkingonintegraNngtheCRMdatawiththatJSONdatawe

stored

Developer


DBA

32Public

Thereisnoneedtointegrateattheapplevel.YoucanuseSQLtorunaqueryagainsttheJSON&theCRMdata.LetmeshowyoubutfirstI’llneedtocheckwhatthe

structureofyourJSONlookslike.

Developer


DataGuide:UnderstandingYourJSONDocuments

• Metadatadiscovery:discoversthestructureofcollecNonofJSONdocuments– OpNonal:deepanalysisofJSONforListofValues,ranges,sizingetc.

• AutomaNcallyGenerates– Virtualcolumns– RelaNonalviews

•  De-normalizedrelaNonalviewsforarrays

– Reports/SynopsisofJSONstructure

33


SQL> SELECT JSON_DATAGUIDE(o.order_details) FROM orders o; JSON_DATAGUIDE(O.ORDER_DETAILS) ---------------------------------------------------------------------- [ {"o:path": "$.date", "type": "string", "o:length": 32 }, {"o:path": "$.order", "type": "array", "o:length": 256 }, {"o:path": "$.location", "type": "object", "o:length": 32}, {"o:path": "$.location.city", "type": "string", "o:length": 8}, .... { "o:path": "$.salesAmount", "type": "number", "o:length": 4 }, { "o:path": "$.loyaltyCardNo","type": "number", "o:length": 8}]

DataGuide:UnderstandingYourJSONDocuments


QueryJSONwithRelaNonalDataSQL> SELECT c.c_name,

o.order_details.location.city,

SUM(To_number(o.order_details.salesAmount)) total

FROM customers c,

orders o

WHERE c.c_custid = o.order_details.loyaltyCardNo

GROUP BY c.c_name,

o.order_details.location.city

ORDER BY total Desc

FETCH FIRST 10 ROWS ONLY;

35

FETCHFIRST10ROWSONLY

•  Newsyntaxtolimitnumberofrowsreturned

•  ReplacesSELECT*FROMWHEREROWNUM<11


Developer DBA

Tomakeiteasierforyou,Icancreateaviewsothedevelopersdon’tneedtoknowallofthe

joincondi=ons

Yeahright,andanybodyis

rememberingthatstuff!


CreaNngaViewonTopofJSON&RelaNonalDataSQL> CREATE OR replace VIEW crm_data AS

SELECT c.c_name,



FROM customers c,

orders o


GROUP BY c.c_name,

o.order_details.location.city

ORDER BY total Desc

FETCH FIRST 10 ROWS ONLY

37


CreaNngaViewonTopofJSON&RelaNonalDataSQL> SELECT *

FROM crm_data;

38


Developer DBA

Noway,thatjustleadstoalotofroundtripsbetweenappanddatabase,whichwillmakethingseven

slower.Andbesides,thisproblemhasalreadybeen

solved.EverheardofMaterializedViews?

That'sgreatbuthowwillthatworkwhenwehaveGBsofdatainthere.Surelyjust

selecNngtheJSONfirstandthenretrievingthe

correspondingcustomerdataisfasterandsaferperformance

wise


Ohboy,thisonereallydoesn’tfeeltheforce!

Amaterializedwhat?

Developer DBA


WhatisaMaterializedView?• Amaterializedviewisadatabaseobjectthatcontainstheresultsofaquery•  Thequerycanbebaseontables,vieworothermaterializedviews•  TypicallyusedtoavoidrepeaNngcostlyaggregaNons• OpNmizertransformsqueriesagainstoriginaltablesintoasemanNcallyequivalentqueryagainstthematerializedview

• AtradeoffbetweendiskspaceandCPU



CreaNngReal-TimeMVonTopofJSON&RelaNonalDataSQL> CREATE MATERIALIZED VIEW top_customers_mv

ENABLE query rewrite

ENABLE on query computation

AS SELECT c.c_name,



FROM customers c,

orders o


GROUP BY c.c_name,

o.order_details.location.city;

42

ENABLEonquerycomputa=on

•  Real-TimeMaterializedView

•  EnsureyoualwaysgetaccurateanswerevenifmaterializedviewisstalebyjoiningtoMVlog


QueriesAutomaNcallyRedirectedtoUseMaterializedView

43

SQL> SELECT *

FROM crm_data;; --------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | --------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10 | 20550 | |* 1 | VIEW | | 10 | 20550 | |* 2 | WINDOW SORT PUSHED RANK | | 1 | 2029 | | 3 | MAT_VIEW REWRITE ACCESS FULL| TOP_CUSTOMERS_MV | 1 | 2029 | ---------------------------------------------------------------------------


I’msNllnotsurethisapproachwillscaleasweneedthequeriestobe

returnedinsub-seconds.Canwetryitwithabiggerdata

set?

Developer DBA

Sure.Whereisthebiggerdataset?


TestMVQueryAgainstLargerDataset3.2millionJSONdocuments


SQL> CREATE MATERIALIZED VIEW top_customers_history_mv

ENABLE query rewrite

ENABLE on query computation

AS SELECT c.c_name,



FROM customers c,

orders_history o


GROUP BY c.c_name,

o.order_details.location.city;


Developer DBA

Thewhat?Ohthat’sgreatfortheknownreportsbutthe

markeNngfolkswanttorunad-hocreportstoo.Whatcanwedotomakesuretheystay

within30seconds?Don’tpanic,wecanput

thetablesintotheIn-Memorycolumn

store


OracleDatabaseIn-Memory:DualFormatArchitecture

•  BOTHrowandcolumnformatsforsametable

•  SimultaneouslyacNveandtransacNonallyconsistent

•  OLTPusesprovenrowformat

•  AnalyNcs&reporNngusenewin-memoryColumnformat•  SeamlesslybuiltintoDatabase•  Allenterprisefeatureswork

-RAC,Dataguard,Flashback,etc47

BufferCache

NewIn-MemoryColumnStore

SALES SALESRow

FormatColumnFormat

SALES


•  FullJSONdocumentspopulatedusinganopNmizedbinaryformat

•  AddiNonalexpressionscanbecreatedonJSONcolumns(e.g.JSON_VALUE)&storedincolumnstore

•  QueriesonJSONcontentorexpressionsautomaNcallydirectedtoIn-Memoryformat

Superfast/MulN-ModelAnalyNcs:In-MemoryJSON

RelaNonal

In-MemoryColumStore

In-MemoryVirtualColumns

In-MemoryJSONFormat

RelaNonal Virtual JSON

48

{"date": "2016-05-03 10:40:36","loyaltyCardNo": 1230, "location": { "city": "Dallas", "state": "TX”}, "salesAmount": 7.55, "order": [ { "item": "Espresso”, …. }}


SQL> ALTER TABLE orders_history ADD city AS (JSON_VALUE(order_details, '$.location.city'));

SQL> ALTER TABLE orders_history ADD state AS (JSON_VALUE(order_details, '$.location.state'));

SQL> ALTER TABLE orders_history ADD salesamount AS (JSON_VALUE(order_details, '$.salesAmount'));

CreateVirtualColumnsontheOrders_Historytable

49


SQL> ALTER TABLE customers INMEMORY;

SQL> ALTER TABLE orders_history INMEMORY;

SQL> SELECT count(*) FROM customers;

SQL> SELECT count(*) FROM orders_history;

SQL> SELECT segment_name,

populate_status, bytes_not_populated

FROM v$im_segments;

PlaceTablesintoIMColumnStore

50


DBA

51Public

Wow,that’sfantasNc.WillitautomaNcallystay

In-Memory?Yup,alldoneforyoubythedatabase!

Thisisawesome!That'sexactlywhatyouwantfrom

adatabase.Schemaflexibility,analyNcsand

performance.

Developer


Heyonemorething,Chewiehasgivenussomeridiculoussecurityrequirements.Nobodyisallowedtoseetheloyaltycardnumberandstufflikethat.Canwedo

somethingtheretoo?"

Developer DBA

SurewecanLet'simplementdata

redacNon.


DefineaRedacNonPolicytoRedactEmployeeLastNameSQL> BEGIN

dbms_redact.add_policy( object_schema => ’coffeeshop',

object_name => ’customers',

column_name => ’c_custid',

policy_name => 'redact_Loyal_crd',

function_type => DBMS_REDACT.FULL,

expression => '1=1');

END;

/



Thisisjustawesome,canyoualsogivemeaREST

endpointtothat?

Developer DBA

Ofcourse!


Man,Iwishwewouldhavetalkedtoyousooner.Youpreoymuchgotourgo-livedownfrom6to1week!Hey,canwejustincludeyougoingforwardinourdiscussionsand

projectplanning?

Developer DBA

I'dbedelightedto!Ialwaysthoughtweshouldworkmuchclosertogether.It'salsogreatformetohavebeoerunderstandwhatappsyouarebuildinginorderto

supportthem.


Summary:OracleDatabase12cUniversalPeaceMaker


Summary:

FlexibilityNaNveJSONsupportRESTServices

ScalabilityScale-outandScale-up

Real-TimeAnalyNcsJSONintheIn-MemoryColumnStore

SecurityDataencryptedatrestDataRedacNon

Oracle Database 12c IT WARS - nloug.nl · PDF filecommitment to deliver any material, code ......

Documents

Transcript of Oracle Database 12c IT WARS - nloug.nl · PDF filecommitment to deliver any material, code ......