Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan...
Transcript of Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan...
![Page 1: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/1.jpg)
Oracle Aware Flash: Maximizing Performance and Availability for your Database
Gurmeet Goindi Principal Product Manager Oracle Kirby McCord Database Architect US Cellular
Kodi Umamageswaran Vice President, Exadata Software Oracle
![Page 2: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/2.jpg)
2 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
![Page 3: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/3.jpg)
3
Program Agenda
Flash and Databases
Oracle’s Innovations in Flash
Customer Case Study – US Cellular
Questions & Answers
![Page 4: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/4.jpg)
4 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
The Promise Of Flash Replace expensive 15K RPM disks with fewer Solid
State (Flash) Devices – Reduce failures & replacement costs – Reduce cost of Storage subsystem – Reduce energy costs
Lower transaction & replication latencies by eliminating seek and rotational delays Replace power hungry, hard-to-scale DRAM with
denser, cheaper devices – Reduce cost of memory subsystem – Reduce energy costs
![Page 5: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/5.jpg)
5 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
The Complexity Of Flash Data loss
– Caused by defects, erase cycles (endurance), and interference Density Vs Endurance
– SLC: Single bit per cell Less density, greater endurance
– MLC: Multiple bits per cell Greater density, less endurance
Write-in-place is prohibitive – Read, Erase, Rewrite: Must be avoided
Low per Chip bandwidth Compensate for these: Complex Firmware
![Page 6: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/6.jpg)
6 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Applications Of Flash In Databases
As an additional storage layer (caching) – Stage active Database objects in flash – Accelerate reads and writes to these objects
For Data Files – Improves user transaction response time – Increases overall throughput for IO intensive workloads
![Page 7: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/7.jpg)
7 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
To Cache Or Not To Cache
Random reads against tables and indexes – Cached: more likely to have subsequent reads
Sequential read tables, or Scans – Not Cached: sequentially accessed data is unlikely to be followed by reads
of the same data Backups, mirrored copies of the block
– Not Cached: Why? But most general purpose flash solutions are
database agnostic and cache all the above workloads
![Page 8: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/8.jpg)
8 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Flash And Database Logs
Flash has very good average write latency Greatly improves user
transaction response time Flash occasional outliers, one
or two orders of magnitude slower
– Garbage collection etc contribute to that delay
OLTP workloads dislike such large variations
![Page 9: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/9.jpg)
9
Where To Introduce A Flash Device Direct Attached
– Mount flash inside your server – PCI-E or SSD
Networked Storage – Share device on the network (FC or 10GE) – Popular implementations: Tiered Storage: Multiple tiers of disk drives (SSD,
FC, SAS); various performance characteristics and data moves between these tiers
Hybrid Storage: Combination of Direct Attached flash on the storage controller and HDD in expansion shelves
All Flash Arrays: All storage is some form of flash device – either SSD or custom Flash modules
NAS/SAN
SSD PCI-E Flash
PCI-E Flash SSD HDD HDD SSD SSD SSD
![Page 10: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/10.jpg)
10
Server Attached Flash
PCI Express attached Flash I/O cards – Extremely fast, improves performance for currently active data – Eliminates round trip between server and storage => Fastest
I/O response – Easy to install and configure, but usually not hot swappable
SSD attached via SATA or SAS – Slower than PCI Express – Cheaper than PCI Express attached flash
SSD
PCI-E Flash
![Page 11: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/11.jpg)
11
Server Attached Flash - Limitations Database limited to size of single server
– Storage limited by number of PCIe or disk slots in the server ANY component failure = loss of database access
– Firmware, Card, Driver, etc. – Local second server with identical Flash recommended ($$$ x 2) – MTBF of the Server decreases as the number of flash cards increases
No Data Loss Requires Synchronous Log Shipping to standby server – Negates performance benefit of Flash writes
Poor resource utilization – Local storage results in islands of storage – No IO resource management for prioritizing multiple databases
![Page 12: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/12.jpg)
12
Tiered Or Hybrid Storage
Hierarchy of storage tiers based on performance The Storage controller moves the data between these layers based
on usage More resilient and scalable architecture compared to server
attached flash Better storage resource utilization as it gets shared among multiple
servers Server attached flash is usually faster then networked attached
flash
![Page 13: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/13.jpg)
13
Tiered Or Hybrid Storage - Limitations
The Storage Controller software determines which data is hot – Has limited visibility to the content of the data – Driven by usage patterns and not application needs – Usage pattern changes as the database workload changes (OLTP Vs
Reporting) makes the achieved performance less predictable – Usually caches data based on yesterday’s workload
Storage controllers are IO bound – Flash can exacerbate that limitation
![Page 14: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/14.jpg)
14
All Flash Arrays No spinning media, all SSD, or proprietary flash modules, or a
combination Best in class performance among storage based solutions Expensive, but innovations in Flash and Storage management are driving
the cost down Limited throughput – usually in low single digit GB/s Most implementations don’t scale very well Availability not at the same maturity level as traditional storage arrays. Are missing functionality that traditional arrays have added over the last
two decades.
![Page 15: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/15.jpg)
15
Putting It All Together
Flash devices in application tier (server attached flash) lacks enterprise class scalability and high availability Flash devices in traditional storage arrays are not efficient as
storage controllers don’t respond quickly enough to workload changes and are IO bound All Flash Arrays lack the features and stability of traditional arrays
![Page 16: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/16.jpg)
16
Program Agenda
Flash and Databases
Oracle’s Flash Architecture and Innovations in Flash
Customer Case Study – US Cellular
Questions & Answers
![Page 17: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/17.jpg)
17 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Oracle’s Flash Architecture
Scale out architecture adds flash capacity and performance by adding
storage servers adds networking and CPU needed to process
flash in one unit Database Aware Storage
Metadata about IO present on the cell Flash on the Storage Server enables sharing
A block on disk is stored in only one flash cache
![Page 18: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/18.jpg)
18 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Oracle’s Innovations in Flash
Exadata Smart Flash Cache Exadata Smart Flash Log Exadata Smart Flash Cache Compression Exadata Smart Flash Cache Scan Awareness Exadata Smart File Initialization
![Page 19: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/19.jpg)
19 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache
Understands different types of I/Os from database
– Skips caching I/Os to backups, data pump I/O, archive logs, tablespace formatting
– Caches Control File Reads and Writes, file headers, data and index blocks
Write-back flash cache – Caches writes from the database not just reads
RAC-aware from day one
![Page 20: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/20.jpg)
20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Log
Outliers in log IO slow down lots of clients Outliers from any one copy of mirror
affect response time Performance critical algorithms like
space management and index splits are sensitive to log write latency Legacy storage IO cannot
differentiate redo log IO from others
foreground
client
Log Buffer
Log writer
foreground
client
foreground
client
foreground
client
![Page 21: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/21.jpg)
21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Log
Smart Flash Log uses flash as a parallel write cache to disk controller cache
Whichever write completes first wins (disk or flash)
Reduces response time and outliers – “log file parallel write” histogram improves – Greatly improves “log file sync”
Uses almost no flash capacity (< 0.1%) Completely automatic and transparent
![Page 22: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/22.jpg)
22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Compression Exadata Automatically Compresses all data in
Smart Flash Cache – Compression engine built into flash card – Zero performance overhead on reads and writes – Logical size of flash cache increases upto 2x
Data Format Compression
Uncompressed Tables 1.3X to 4X
Indexes 1.3X to 4X
Oracle E-biz uncompressed DB 4X
OLTP Compressed Tables 1.2X to 2X
HCC Compressed Tables Minimal
Great for
OLTP
![Page 23: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/23.jpg)
23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Compression
Flash Device
User Data
Flash Media Data
As user adds more data, data is compressed and written Flash device has no logical space at the end for user data but
has lot of physical space
![Page 24: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/24.jpg)
24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Compression
Flash Device
User Data
Flash Media Data
Extend the logical address space to store more data
![Page 25: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/25.jpg)
25 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Compression
Flash Device
User Data
Flash Media Data
In reality, data doesn’t compress with a uniform compression ratio If a new block (green) update compresses more than previous
block (red) there is nothing to do, there is more free space
![Page 26: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/26.jpg)
26 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Compression
Flash Device
User Data
Flash Media Data
If new block (green) does not compress as well as the previous block (red) Run out of space? That should never happen!
![Page 27: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/27.jpg)
27 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Compression
Flash Device
User Data
Flash Media Data
Exadata periodically monitors free space Performs TRIMs on blocks to ensure enough free space for all IOs Space reclaimed via LRU tail – not the next block TRIMs show up as writes (watch for it in iostat) Device size shown in Linux much larger than 2x
TRIM block
![Page 28: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/28.jpg)
28 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Compression Exadata automatically Compresses all data in
Smart Flash Cache – Compression engine built into flash card – Zero performance overhead on reads and writes – Logical size of flash cache increases upto 2x – User gets large amount of data in flash for same
media size – Enabled via “cellcli –e alter cell flashCacheCompression=TRUE”
– Elasticity of flash cache is completely automatic and transparent
![Page 29: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/29.jpg)
29 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Scan Awareness
On a traditional cache, if you scan dataset larger than cache size
– Blocks 0,1,2,3 brought into cache, cache is full – Block 20,21,22,23 say replaces 0,1,2,3
Repeat the same scan – Block 0,1, 2, 3 will replace blocks 20,21,22,23 – Block 20,21,22,23 will again replace block 0,1,2,3
Traditional caches churn with no actual benefit Some implementations call the insertion of new
block in the middle scan resistant
LRU
Insert new block
Churn
![Page 30: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/30.jpg)
30 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Cache Scan Awareness Exadata Smart Flash Cache is scan resistant
– Ability to bring subset of the data into cache and not churn – OLTP and DW scan blocks can co-exist
Nested scans bring in repeated accesses – Repeat, For each item in large table, scan small table – Smart enough to pull the small table into flash since it is
accessed repeatedly even though the size of large table alone is larger than flash cache
No need to set “KEEP” attribute in data warehouses Happens automatically, no tuning or configuration
needed
Cache
![Page 31: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/31.jpg)
31 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart File Initialization File Initialization Write Back Flash Cache
11.2.3.2 Smart File Initialization
11.2.3.3
CELLSRV
Database
Metadata
I/Os in parallel
CELLSRV
Database
I/O
I/O optimized
CELLSRV
Database
Metadata
I/O optimized
![Page 32: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/32.jpg)
32 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart File Initialization Combine the benefits of previous Smart Initialization
and Writeback Flash Cache – Write file creation meta-data to writeback flash cache – Tiny amount of flash space used to cache large portions
of initialized data on disk – File creation sped up by an order of magnitude – Initialization I/Os to disk deferred or not performed if data
loaded
Create tablespace, file extensions, autoextend show benefit Redo log initialization included in Exadata 12.1.1.1.0 Happens automatically, no tuning needed
CELLSRV
Database
Metadata
I/O optimized
![Page 33: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/33.jpg)
33 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Exadata Smart Flash Benefits
Smart Flash Cache is database aware Smart Flash Logging avoids redo log outliers Smart Flash Compression doubles flash media
capacity Smart Flash Cache Scan provides subset scannning
and is table scan resistant Smart File Initialization creates a file but just writing
meta-data to flash cache Happens automatically, no tuning needed
![Page 34: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/34.jpg)
34
Program Agenda
Flash and Databases
Oracle’s Innovations in Flash
Customer Case Study – US Cellular
Questions & Answers
![Page 35: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/35.jpg)
Oracle Database Kirby McCord Lead Integration Infrastructure Architect
![Page 36: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/36.jpg)
36
USCellular United States Cellular Corporation provides a comprehensive range of wireless products and services, excellent customer support, and a high-quality network to 5.0 million customers in 23 states. The Chicago-based company had 7,000 full- and part-time associates as of June 30, 2013.
![Page 37: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/37.jpg)
37
Who am I?
Lead Infrastructure Architect at USCellular Oracle OCP since Oracle 7
Member of the IOUG Conference Committee • Infrastructure Track Member 2011 • Infrastructure Track Lead 2012 • Engineered System Manager 2013
IOUG Collaborate Speaker 2012 & 2013
Contributor to IOUG’s Exadata Tip Booklet
![Page 38: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/38.jpg)
38
Exadata System
ZFSSA 7410 Storage Array
X2-2 Storage Expansion (HC Disks)
X2-2 Full Rack (HP Disks)
ODS
EDW
DW
Misc
4 Databases ODS – Primary loaded via GoldenGate, some batch loading. Mixed workloads on access. Majority of workload on server. EDW – Warehouse with atomic layer in 3NF with some star schema built in DW – Traditional Warehouse, star schema. Misc – Small adhoc workload
All Databases are • Multi-instance • Have Physical Standby’s • Backed up to the ZFFSA
![Page 39: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/39.jpg)
39
Previous Solution – ODS Only 10gR2 Database built around Oracle’s Reference Architecture Monolithic Unix Server • 24 cores, • 384 GB RAM • 10gE Networking • 8 8G FC SAN connectivity Enterprise Grade Storage Array • 15K SAS drives (short armed disks) • Lots of cache • Dedicated to Database
Note: The 2 solutions had similar architectures, loaded via GoldenGate and Informatica and mixed workload applications accessing. The Exadata based system is from a newer billing system, which is 2-3x the amount of changes to load via GG. We also have a lot more applications accessing the ODS. The new workload is larger than what the previous hardware could handle.
![Page 40: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/40.jpg)
40
What Did We See - Exadata ODS
1.49 ms single block reads While doing 42K read IOPS and 11K write iops over an hour period.
Note: The other databases were active on the Exadata System during this time.
What? Writes are supposed to be fast! Wait until later slides.
![Page 41: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/41.jpg)
41
Comparison to Old system Metric Exadata ODS Monolithic
Hardware ODS Comparison
Single Block Reads 1.5 ms 3.8 ms > 2x
Log File Synch Waits
.85 ms 5.7 ms > 6x
Note: The Exadata ODS is over twice the workload as the previous version. In addition, the Exadata system is shared with several databases, while the Monolithic Hardware was dedicated.
![Page 42: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/42.jpg)
42
Write Back Flash Enablement Design to accelerate write intensive workloads. From previous slide, we had lots of “free buffer waits”. Enabled this feature on X2-2. Result: No more “free buffer waits”.
Writes I/Os
![Page 43: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/43.jpg)
43
Maintenance Activities Preventative Maintenance Operations • Battery replacement • Proactive disk replacement
Unplanned Operations • Disk Replacement • Flash Drive Replacement • Motherboard Replacements
Patching • All patches for over a year…
All Have Been Done in a Rolling Fashion!
![Page 44: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/44.jpg)
44
What This Means to Us More Flexibility in System Use • We are less concern about unplanned activities on the system. The users
can go after the system when they need to, not during certain windows. • Maintenance activities have less impact on system availability.
More Use of the Data • Exadata’s Flash reduces the i/o contention of the mixed workloads within
the database and between competing databases • More concurrent users mean more business questions being answered.
Faster Access to the Data • Faster I/O means less time waiting for queries to return, more time to
analyze the results
![Page 45: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/45.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 45
Program Agenda
Flash and Databases
Oracle’s Innovations in Flash
Customer Case Study – US Cellular
Questions & Answers
![Page 46: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/46.jpg)
46
Graphic Section Divider
![Page 47: Oracle Aware Flash: Maximizing Performance and ...€¦ · On a traditional cache, if you scan dataset larger than cache size – Blocks 0,1,2,3 brought into cache, cache is full](https://reader036.fdocuments.us/reader036/viewer/2022071013/5fcb80e9fbbeab064c34bd51/html5/thumbnails/47.jpg)
47