Post on 03-Nov-2019
© Copyright 2016. Apps Associates LLC. 1
Experience of being a Cloud DBA
Satyendra Pasalapudi Associate Practice Director Apps Associates LLC
April 26, 2016
© Copyright 2016. Apps Associates LLC. 2
About Me
Satyendra Kumar Pasalapudi
Associate Practice Director – IMS & Cloud Practice at Apps Associates
Co-Founder & President of AIOUG
@pasalapudi
© Copyright 2016. Apps Associates LLC. 3
About Apps Associates
• Global Reach for Global Customers
– Founded in 2002, 650+ employees
– US, Europe, Asia
– US (East, Central, West)
– HQ in Boston, Massachusetts
• Strategic Partnerships, Certifications, Credentials
– Oracle Platinum Partner – Oracle Specialized Across Our Portfolio of Services
– AWS Premier Consulting Partner – Oracle and Managed Service Partner Competencies
– Salesforce Cloud Alliance Partner
– MuleSoft Partner
– CMMI Level 3 Appraised
• Cloud Services – Majority Investment in Pre-Sales, Demand Gen, Labs & Solutions
© Copyright 2016. Apps Associates LLC. 4
Agenda
• What is Cloud
• Virtualization the mother of Cloud
• Oracle on Cloud
• Storage Considerations
• Customer Case Study
© Copyright 2016. Apps Associates LLC. 5
Cloud
© Copyright 2016. Apps Associates LLC. 6
Generational Change for Enterprise (IT)
Cloud supports mission critical workloads ─ 87% of Enterprises use Cloud for Mission
Critical Applications
Cloud use in the enterprise continues to
grow ─ Half of the Enterprises say they will use
cloud for at least 75% of their workloads by 2018
No one cloud fits all
─ More than half (53 %) of enterprises use two(2) to four(4) cloud providers
Source: Verizon 2016 State of the Market: Enterprise Cloud report
© Copyright 2016. Apps Associates LLC. 7
Cloud – Probable to Inevitable
GE undergoing most important transformation in 140 year history ─ 9000 Applications to AWS & to 4000 Applications
─ 300 ERPs (two years back) to more manageable
─ 34 Data Centers to 4 Data Centers
By 2020 - US$15b of Software Revenue
Changes ─ People - Reduce Outsourcing
─ Technology - Build Approach for things that matter
─ 20% of Applications in Cloud as of today
─ 70% of Applications by 2020 in Cloud
Source: AWS 2015 Keynote – Oct 6 2015
OOW Keynote with Mark Hurd Oct 26 2015
─ Service Management ─ Network Perimeter ─ Risk Based Security Controls ─ Self Service and Automation ─ Financial Transparency
© Copyright 2016. Apps Associates LLC. 8
Hybrid Cloud Framework
HR FIN
SCOM SALES
PROCUREMENT
PLANNING
DW / BI
© Copyright 2016. Apps Associates LLC. 9
What is Cloud
© Copyright 2016. Apps Associates LLC. 10
What is Virtualization? • It is a abstraction of computer hardware resources
Virtualization technology emulates real — or physical — computing
resources, such as desktop computers and servers, processors
and memory, storage systems, networking, and individual
applications.
A hypervisor — also known as a virtual machine manager
(VMM) — allows multiple “guest” operating systems
to run concurrently on a single physical host
computer.
Types Hardware Virtualization Full Software Virtualization Para virtualization Hardware-assisted software
virtualization Component or resource
virtualization
Virtualization players in market
© Copyright 2016. Apps Associates LLC. 11
Reasons for Virtualization?
Server Consolidation
Server Provisioning
Backup/Restore
Hosting/ Cloud Computing
© Copyright 2016. Apps Associates LLC. 12
Types of Hypervisors
Oracle VM VMware vSphere KVM Xen Hyper-V
VirtualBox VMware Fusion VMware Player VMware Workstation
Guest OS
Guest OS
Guest OS
Server
Hypervisor
Type 1 – Native (Bare metal)
Guest OS
Guest OS
Guest OS
Operating System
Hypervisor
Server
Type 2 – Hosted
© Copyright 2016. Apps Associates LLC. 13
Virtualization in a nutshell
Virtualization : Running multiple machines on a single hardware
“Real” hardware invisible to OS
OS only sees an abstracted out picture
Only Virtual Machine Monitor (VMM) / Hypervisor talks to hardware
Traditional
Environment
Virtualized
Environment
© Copyright 2016. Apps Associates LLC. 14
Pros and Cons – Full Virtualization
• Pros
– Disaster recovery, failover
– Virtual appliance deployment
– Legacy code on non-legacy hardware • Cons – LATENCY of core four resources
– RAM performance reduced 25% to 75%
– Disk I/O degraded from 5% to 20%
– Network performance decreased up to 10%
– CPU privileged instruction dings nearing 1% to 7%
© Copyright 2016. Apps Associates LLC. 16
Database I/O Patterns
• Cache data in memory from disk to achieve fast query response
• For OLTP/ DSS workloads, memory should be big enough to hold as much as possible
SGA
Buffer Cache
100% cache hit ratio is ideal
for OLTP workloads
© Copyright 2016. Apps Associates LLC. 17
Database I/O Patterns
• CPU time is mainly consumed by I/O wait
Cache miss Cache Hit
© Copyright 2016. Apps Associates LLC. 18
Database I/O Patterns
2. User Volume is
increasing...
1. Data Size is increasing...
SGA
1. Huge amounts of data
2. Not possible to cache
all data
3. Many I/O operations
Buffer Cache
© Copyright 2016. Apps Associates LLC. 19
Cloud Evaluation Considerations
Running Oracle On Cloud
© Copyright 2016. Apps Associates LLC. 20
Key Considerations for running on Oracle Cloud
Oracle Database
Security
Compliance
Hybrid Cloud Connectivity
Licensing & Supportability
Partner Ecosystem
Performance
© Copyright 2016. Apps Associates LLC. 21
Enterprise Class Oracle Database Architecture
© Copyright 2016. Apps Associates LLC. 22
Architectural Considerations • Storage Scalability
– SSD / PIOPS
– Striving to match SAN like performance
• Compute Scalability
– Vertical scaling – massive CPU cores
– Horizontal Scaling – clusters
– Oracle RAC on Cloud
• Network Scalability
– Bandwidth constraints ( With in the Cloud )
– Leased lines to Cloud ( Hybrid Cloud )
© Copyright 2016. Apps Associates LLC. 23
AWS Storage Options Summary
Storage Option General Database Applications Speed Durability
Instance Store Swap Temporary files Reports cache,
web server cache Very low latency Very low; volatile
EBS—PIOPS / SSD
Data files, redo
logs Low latency
Highly Durable but always
backup
EBS Boot volume Binaries, archive
logs Binaries Moderate latency
Highly Durable but always
backup
Amazon S3 Backups Backups Backups Longer latency Very high durability
Amazon Glacier
Long-term
backups Long-term backups Long-term backups
Restore times of
3–5 hours Very high durability
© Copyright 2016. Apps Associates LLC. 24
Storage Best Practices for Oracle
• Oracle Automatic Storage Management feature is the best option to optimize storage performance on AWS
• SAME (Stripe And Mirror Everything) feature of Oracle ASM will help distributing entire data across multiple ebs volumes
• ASM along wit PIOPS EBS volumes can provide best in class storage performance on AWS
• Oracle Secure Backup for Cloud Module will enable backups directly from Oracle Database/ASM to S3 using RMAN
• RMAN channels can be optimized for optimal system throughput and Storage performance
© Copyright 2016. Apps Associates LLC. 25
+DEV_DATA
4 EBS Volumes 500 PIOPS
DEV TEST
Storage Options for Oracle
Sample disk layout for Oracle Database using ASM
Amazon S3 Bucket
Amazon Glacier
Life Cycle Policies
Striping
+PROD_DATA
4 EBS Volumes
4000 PIOPS
+PROD_FRA
4 EBS Volumes
1000 PIOPS
PROD
Striping Striping
ASM instance with 16,000+ IOPS
© Copyright 2016. Apps Associates LLC. 26
Typical Storage Config for Oracle Database
Storage Type Comments
Binaries EBS volume Standard volume; EBS snapshots enabled
Data Files PIOPS—EBS / SSD volumes
Striped across multiple volumes using ASM or any other technology PIOPS of 1000 or above
Redo Log Files PIOPS—EBS / SSD volumes
Use separate EBS volumes for each group Preferable to use ASM disk groups
Archive Log Files EBS volumes Standard volumes for normal database workloads PIOPS for highly transactional environments
Backup Files EBS volumes / Amazon S3
Standard EBS volumes for local backups Use OSB / other technology to push to Amazon S3
© Copyright 2016. Apps Associates LLC. 27
Backup on Cloud
– Use Amazon S3 for all backups
– Snapshot backup mechanism non-data file volumes
– Use RMAN to backup the database to Amazon S3 using Oracle Secure Backup Cloud Module (OSB)
– Archive data to Amazon Glacier for long term storage
Huge Pages
Huge Pages
Huge Pages
Huge Pages
3
3
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K
4K 4K 4K 4K 4K 4K 4K 4K
4K 4K
4K 4K
2 MB
2 MB
2 MB 2 MB 2 MB
72 GB RAM 24 GB
RAM
48 GB
HugePages
Default
HugePages pool
Huge Pages
Advantages of HugePages HugePages are not swappable, in other words they are always pinned, so no swapping overhead. HugePages can be allocated on-the-fly but must be reserved first by setting the kernel parameters HugePages are reserved at system startup AMM (Automatic Memory Management) and HugePages
are not compatible, AMM should be disabled so that
Database can utilize HugePages. 11gR2 logs HugePages information in alert.log.
Huge Pages on Cloud
Under the covers, Amazon EC2 uses a hypervisor to (potentially) run many virtual
machines on a given physical server. EC2 instances essentially come in two flavors
of virtualization:
Para virtualization (PVM) and Hardware virtualization (HVM).
The vast majority of EC2 AMIs use PVM, but for a variety of reasons, only EC2
instances using HVM can allocate huge pages.
Evolution of Virtualization Types • 1st Generation: Full
virtualization (Binary
rewriting)
– Software Based
– VMware and Microsoft
Time
• 3rd Generation: Silicon-
based (Hardware-assisted)
virtualization
– Unmodified guest
– VMware and Xen on
virtualization-aware
hardware platforms
• 2nd Generation: Para
virtualization
– Cooperative
virtualization
– Modified guest
– VMware, Xen
Dynamic Translation
Virtual
Machine
Hardware
Operating System
Virtual
Machine …
Hypervisor
Hardware
VM VM
Hardware
Virtual
Machine
Virtual
Machine …
Virtualization Logic
Hypervisor
…
*This slide is from Intel® Corporation
Smart Flash Cache
How Smart Cache Works
© Copyright 2015. Apps Associates LLC. 39
Customer Case Study
Customer Case Study
• Oracle E-Business Suite R12.1.3 – Oracle Database 11g (11.2.0.3)
• Database size – 1.5 TB
• OEL 5.8 64-bit
• 24 cores with 98GB RAM x 2 Nodes
• 11.2.0.3 RAC on ODA
– 2 - Middle tier application nodes
• 8 cores with 16Gb RAM
• RHEL 6.3 64-bit
Primary Datacenter
Pro
du
ction
D
B Tie
r
DB 11g
Ap
p Tier
Customer via Internet
App Tier
Customer WAN
© Copyright 2015. Apps Associates LLC. 41
Key Considerations
AWS Sizing – Compute & Storage
• Right sizing Storage IOPS • Instance type selection • RAC vs Non-RAC • Oracle ASM
Shared Storage • Sharing batch job log &
out files across applications
• Avoiding NFS as single point of failure
Oracle Specific Techniques for Migration
• Oracle Data Guard for incremental data transfer
• Oracle Flash Cache with ephemeral SSD volumes
RTO / RPO Considerations • Build Data Guard based
local DR for better RTO • Elastic Load Balancers
with EBS
© Copyright 2015. Apps Associates LLC. 42
Cloud Architecture
Primary Datacenter
Customer WAN
Pro
du
ction
D
B Tie
r
DB 11g
Ap
p Tier
Customer via Internet
App Tier
AZ-1
Private Subnet
US East
App
Node
1
DB Tier
M
App
Node 2
Private Subnet
DB Tier
S
App
Node
3
App
Node 4
AZ-2
Oracle DataGuard
© Copyright 2015. Apps Associates LLC. 43
Key Considerations
• Current Production DB on RAC cluster with 2 nodes
• On AWS, DB tier will be migrated to single node (No RAC)
• To ensure RTO / RPO guidelines, cross AZ DB high availability needs to be built
using Data guard
• Application tier high availability should also be considered in proposed design
• Overall solution to be optimized as per AWS high availability guidelines
© Copyright 2015. Apps Associates LLC. 44
On Premise Database Tier Configuration
• Source environment
– Oracle Database Appliance
– Internal storage for Database files on ASM ; SSD for REDO log files
– Backup storage from SAN
– Average CPU utilization < 25% ( out of 24 cores)
– Instance caging implemented to limit DB CPU utilization to 8 cores
• Database Configuration
– sga_target – 18G
– pga - 10G
– Total memory allocation 56G across two nodes
© Copyright 2015. Apps Associates LLC. 47
Database Instances – On Cloud
Instance Family Memory Optimized
Instance Type r3.4xlarge
Processor Arch 64-bit
Physical Processor Intel Xeon E5-2670 v2
vCPU ( Cores ) 16
Memory 122GB
SSD ( Ephemeral ) 320GB
EBS – Optimization Available (2000 Mbps)
Network Performance High
Virtualization HVM Only
Oracle Flash Cache feature availability Yes
Price Performance (on-demand) $1.400 std price
Ref: http://aws.amazon.com/ec2/instance-types
© Copyright 2015. Apps Associates LLC. 48
DB Server Config
• DB Server – r3.4xlarge with EBS Optimization
• Storage
– OS Disk – 50G - Root (depends on AMI) (Magnetic/SSD)
– Oracle Binaries – 100G (Magnetic)
– Backup disk - 1 TB – SSD EBS Volume
– Concurrent Log & Out stored on DB tier with NFS to App tiers - 200G SSD
• Automatic Storage Management
Production Non-Prod
DATA (2TB / instance) 1TB x 2 SSD Disks
750 GB x 3 SSD Disks
Or
1TB x 2 SSD Disks
REDO 50GB x 1 Disk with 1000 PIOPS 50GB x 1 Disk SSD
FLASH (Archives) ( <200G/day) 250GB x 1 SSD 250GB x 1 SSD
© Copyright 2015. Apps Associates LLC. 49
Oracle EBS Architecture Summary on Cloud
Role Instance Type AZ CPU Cores Memory
EBS Database Tier r3.4xlarge Primary 16 122
EBS Local DR Tier R3.4xlarge/m3.xlarge DR 16/ 4 122/15
Application Node 1 M3.xlarge Primary 4 15
Application Node 2 M3.xlarge Primary 4 15
Application Node 3 M3.xlarge DR 4 15
Application Node 4 (Passive) M3.xlarge DR 4 15
Load Balancer ELB
© Copyright 2015. Apps Associates LLC. 50
Cloud DBA Core DBA
75% Network
Admin 5%
Storage Admin 15%
Security Admin 5 %
= + + +
© Copyright 2015. Apps Associates LLC. 51