Big DataFrom Deployment to Operations

wouter.jagers@exitas.be

Introducing myself

Big Data and myself

Topics

● DIY vs Engineered Systems● Introducing the Oracle BDA● Startup & growth● Management, monitoring and support● Upgrades and extensions● Experiences in deployment & operations

Big Choices

● Manual Hadoop installation– everything from scratch

● Cloudera stack– all software components + Cloudera Manager

● Big Data Appliance– all-in: hardware + software + CM Enterprise

DIY - Big Decisions

● Servers and Operating System● Storage● Networking and interconnects● Software versions● Configuration and performance tuning● Security and auditing

Big Data Appliance X6-2

Sun Oracle X6-2L Servers with per server:● 2 * 22 Core (2.2GHz) Intel Xeon E5-2699 v4 Processors● 256 GB DDR4-2400 Memory● 96TB Disk space

Included Software (4.5):● Oracle Linux 6.7● Oracle Big Data SQL 3.0.1*● Cloudera Distribution of Apache Hadoop 5.7 – EDH Edition● Cloudera Manager 5.7● Oracle R Distribution● Oracle NoSQL Database CE

Easy start, flexible growth

● BDA starter rack: 6 nodes● In-rack extensions: up to 18 nodes

– one-node extensions are now possible

● Scale up to 18 racks

Start developing right away

● Site-prep● Names and IP addresses, generate config● Deployment● Ready for use !

From installation to actual use in days.

BDA - Big Advantages

● Infiniband interconnects (40Gb/s)

● Integrate Exadata in the same IB network; Big Data Connectors

● 2 to 16 10GB/s client network connections per rack

● Configured and tuned out of the box; always highly available

● Security: Kerberos, Sentry

● Cloudera Manager Enterprise

● Oracle ASR / Enterprise Manager / mgmt tools

● Supported and tested upgrades

CM Enterprise

● Aggregate UI

● Rolling upgrades

● Configuration versioning and history

● Data encryption

● User roles

● File browsing, search, quota management

● Auditing - Cloudera Navigator

● … and more

A Cost Effective Solution“Oracle Big Data Appliance is an excellent choice

for customers looking to work with the full suite of Cloudera’s leading Hadoop-based technology. It’s more cost-effective and quicker to deploy than a DIY cluster.”

Mike Olson, Cloudera founder, Chief Strategy Officer, and Chairman of the Board21%

Cost Savings

33%Faster

Time to Value

Source:ESG White Paper

BDA - Operational tools

● mammoth– deployment

– upgrades

● bdacli– post deployment configuration

– adding / removing components

● health checks and functionality testing– hardware

– software

– hadoop cluster functionality checks

BDA Upgrades

● Review of documentation & known issues● Health checks & cluster validation● Patch / image downloads● Upgrade using Oracle's mammoth

● Downtime < 1 day, or rolling upgrade

In summary - our experiences

● Preparation and installation● Upgrades● Support: Exitas & Oracle● Customer satisfaction

Oracle's BDA allows us to support more clusters in less time.

Thanks !

wouter.jagers@exitas.be