paced lesson entitled “Introducing Oracle R Enterprise ... · Introducing Oracle R Enterprise 1.4...

Introducing Oracle R Enterprise 1.4 -

Hello, and welcome to this online, self-paced lesson entitled “Introducing Oracle R

Enterprise.” This session is part of an eight-lesson tutorial series on Oracle R Enterprise.

My name is Brian Pottle. I will be your guide for the next 45 minutes of interactive lectures and

review sessions on this lesson.

1

Introducing Oracle R Enterprise 1.4 - 2


“Introducing Oracle R Enterprise” is the first lesson of eight self-study sessions on Oracle R

Enterprise.

4


In this lesson, you’ll learn:

• What R is, who uses it, and why they use it.

• Next, we’ll examine several common user interfaces for R.

• Finally, you’ll learn about Oracle’s strategy for supporting the R community.

So, let’s start with the first topic: Using R: What, Who, and Why?

5


R is a language and environment for statistical computing and graphics. This GNU Project is

similar to the S language and environment, which was developed at Bell Laboratories

(formerly AT&T, now Lucent Technologies) by John Chambers and colleagues. R can be

considered a different implementation of S. There are some important differences, but much

code written for S runs unaltered under R.

R is an open-source language and environment that supports:

• Statistical computing and data visualization

• Data manipulations and transformations

• And sophisticated graphical displays

6


With over 2 million R users worldwide, R is increasingly being used as the statistical tool in

the academic world. Many colleges and universities worldwide are using R today in their

statistics classes. In addition, more and more corporate analysts are using R.

R benefits from around 5000 open-source packages, which can be thought of as a collection

of related functions. This number grows continuously with new packages submissions from

the R user community.

Each package provides specialized functionality in such areas as bioinformatics and financial

market analysis.

In the slide, the list on the right shows “CRAN Task Views.” CRAN stands for the

Comprehensive R Archive Network, which is a network of FTP and web servers that store

identical, up-to-date versions of R code and documentation.

The CRAN Task Views list areas of concentration for a set of packages. Each link contains

information that is available on a wide range of topics.

7


So, why do statisticians and data analysts use R?

• As mentioned previously, R is a statistics language that is similar to SAS or SPSS.

• R is a powerful and extensible environment, with a wide range of statistics and data

visualization capabilities.

- Powerful: Users can perform data analysis and visualization with a minimal

amount of R code.

- Extensible: Users can write their own R functions and packages that can be used

locally, shared within their organizations, or shared with the broader R community

through CRAN.

• It’s easy to install and use.

• And it’s free and downloadable from the R Project website.

8


Although it’s a powerful and effective statistical environment, R has limitations.

First, R was conceived as a single-user tool that was not multithreaded. The client and server

components are bundled together as a single executable, much like Excel.

• R is limited by the memory and processing power of the machine on which it runs.

• Also, R can’t automatically leverage the CPU capacity on a user’s multiprocessor laptop

without special packages and programming.

Second, R suffers from another scalability limitation that is associated with RAM.

• R requires data that it operates on to be first loaded into memory.

• In addition, R’s approach to passing data between function invocations results in data

duplication. This “call by value” approach to parameter passing can use up memory

quickly.

So inherently, R is really not designed for use with big data.

Some users have provided packages to overcome some of the memory limitations, but the

users must explicitly program with these packages.

9


R provides a wealth of resources to help users, including:

• Many R-related books that are available on the R project website

• Many user groups and user conferences that are available to the R community

• Online libraries of reusable code from the CRAN website

• Documented R packages with sample data and code

10


Next, let’s examine several common user interfaces for R.

11


First, let’s take a quick look at the interface that comes with open-source R by default, called

the R Console.

• This default open-source R graphical user interface (GUI) includes a command-line

interface for running scripts or individual functions, as shown in the slide.

• In addition, open-source R supports many third-party graphics packages.

- In this example, we load a popular third-party graphics package named “ggplot2.”

- Then, the graphics package is called from the R Console command line. The

second qplot function call displays the graphic on the right.

- Here, the qplot function is invoked on the mtchars data set, which comes with R. In

the graph, we plot miles per gallon against weight, with the size of each dot

indicating the number of cylinders.

12


In addition to the default open-source R GUI, you can use a third-party integrated development environment (IDE), such as RStudio, which is shown in the slide.

With RStudio:

• You can use the upper-left pane to view R scripts and select portions of an R script for execution.

• In the Console pane, you can execute R scripts or functions at the command line, in a similar fashion as the default R GUI.

• You can execute selected portions of R scripts in the top window by clicking the Run button. With this method, selected lines are pasted into the Console pane and executed.

• You can view graph results in the right pane. In this case, the Plots tab is selected.

In this next view, the R script that we saw previously is displayed in the viewer window.

• Here, the first portion of the script is selected. This code requests help on the gplot() function.

• When the Run button is clicked, the selected code is pasted into the Console pane and then executed.

• In the display pane, you can select (and switch between) different tabbed output views on the Files, Plots, Packages, and Help tabs. In this case, the Help tab is selected to display results from the R help command.

In this final view, the last function in the R script is selected.

• This same gplot() function was shown previously in the default R GUI.

• The Run button is clicked and the code is executed. The Plots tab shows the current output. In fact, RStudio also lets you view previously generated plots.

RStudio is only one of many third-party R IDEs.

As shown in the table of this 2011 poll, RStudio is the second most commonly used interface, behind the built-in R console we looked at earlier. However, it’s often user preference that decides which IDE will be used.

13


Data visualization helps convey information faster than most other means. The link shown in

the slide is for the R Graph Gallery, where you can find a variety of graphic types for R.

14


Here are a few examples of graphs in R. Of course, there are many others. Moving from left to

right, and top to bottom, we show:

• A box plot

• Perspective graphs of mathematical surfaces

• 3-D scatter plots with points

• A regression plane

• Multivariate facet crafts

• Smooth scatter plots

• Venn diagrams

• And even chromosome mappings from the bioconductor package

15


In this final section of the lesson, you’ll learn about Oracle’s strategy for supporting the R

community. This section includes the following topics:

• Goals

• Software term definitions

• High- and mid-level architectural overviews

• Software component features

• R user-community definitions

16


Scalability, performance, and production deployment are key requirements for the enterprise

data analytics arena.

• What may work fine on your laptop for thousands or even millions of rows, won’t scale to

100s of millions and billions of rows.

• Similarly, performance may be adequate on smaller scale data, but will moving to “big

data” allow you to keep up? The cost of moving data to a separate server can render an

application unusable.

• Finally, when you’ve finished your project in a lab environment, how easy is it to deploy

that result into production?

Each of these requirements are met by Oracle’s strategy for supporting the R community.

17


Oracle’s goal for supporting open-source R is to deliver enterprise-level advanced analytics

based on the R environment. The strategy is implemented through the release of the following

Oracle technologies:

• Oracle R Enterprise (ORE), which is part of the Oracle Advanced Analytics option for

Oracle Database 12c and 11g, release 2. ORE contains a statistics engine, and

provides transparent access to database-resident data from R, as you will learn in this

tutorial series.

• Oracle R distribution, which supports configurations of open-source R on various

platforms. In addition, Oracle contributes bug fixes and enhancements to open-source

R.

• ROracle, the open-source Oracle database interface for R.

• Oracle R Advanced Analytics for Hadoop, or ORAAH, provides an R interface to an

Oracle Hadoop cluster on the BDA ,and also to non-Oracle Hadoop clusters. It enables

you to access and manipulate data in the Hadoop Distributed File System, in the Oracle

Database, and on the file system.

18


Now, let’s examine an architectural view of ORE.

• The R workspace console may be the default R GUI or any of the third-party R GUIs.

Users execute R scripts here.

• Then, the ORE transparency layer intercepts functions that operate on database tables

or views. It translates the request into SQL for execution in Oracle Database for

transformations and statistical computations. In Oracle Database, the statistics engine

consists of native database functionality that leverages SQL and the various database

management system (DBMS) packages, as well as enhancements that are specific to

ORE.

• Finally, the results can be leveraged by enterprise systems, such as Oracle Business

Intelligence Enterprise Edition (OBIEE), or web services-based applications.

This design results in:

• No changes to the R user experience in the development environment

• The ability to scale to large data sets in the production environment

• And, the ability to embed results in operational systems, such as Oracle OBIEE

Dashboards

19


This architectural view illustrates how ORE can work with Oracle R Advanced Analytics for

Hadoop. ORAAH enables native R access to the Hadoop cluster for both:

• MapReduce programming in R, and

• Access to Hadoop Distributed File System (HDFS) data, in either the Big Data

Appliance (as shown in the slide), or non-Oracle Hadoop clusters

20


Oracle Big Data Appliance has been mentioned a couple of time so far in this lesson. So,

what is it? Oracle BDA:

• Is an optimized solution for storing and integrating low-density data into Exadata.

• Is a preintegrated configuration with 18 of Oracle's Sun servers that include InfiniBand

and Ethernet connectivity to simplify implementation and management.

• Has the Cloudera distribution, including Apache Hadoop to acquire and organize data,

along with Oracle NoSQL Database Community Edition to acquire data.

• Includes additional system software: Oracle Linux, Oracle Java Hotspot Virtual Machine,

and an open-source distribution of R.

Oracle Big Data Connectors is an option for BDA. It consists of:

• Oracle Loader for Hadoop

• Oracle Data Integrator Application Adapter for Hadoop

• Oracle Direct Connector for HDFS

• Oracle R Advanced Analytics for Hadoop

You can use ORAAH to access data in Exadata, and perform R calculations on HDFS data by

using scalable map-reduce methods.

21


Now, let’s take a brief look at the components of Oracle R Enterprise. From a software

perspective, ORE consists of R packages, database libraries, and SQL extensions.

We’ll divide the features into three main groups: the Transparency Layer, the Statistics

Engine, and SQL extensions.

The Transparency Layer is a set of packages that map R data types to Oracle Database

objects.

• This feature automatically generates SQL for R expressions on mapped data types,

enabling direct interaction with data in Oracle Database while using R language

constructs.

• Functionally, this mapping provides access to database tables from R as a type of

data.frame: a base R data representation with rows and columns. ORE calls this an

“ore.frame.”

• Therefore, when you invoke an R function on an ore.frame, the R operation is sent to

the database for execution as SQL.

The Statistics Engine is a database library that supports a variety of statistical computations.

This engine includes existing in-database advanced analytics and new features added

specifically in ORE.

SQL extensions enable in-database embedded R execution, which is particularly valuable

for third-party R packages, or custom functions, that do not have equivalent in-database

functionality.

These SQL extensions also facilitate embedding R in operational systems.

• By executing in the database, you can load data to the R engine more efficiently than by

22


If we look at ORE from the perspective of a collaborative execution model, it leverages three layers of computational engines.

The first one is the client (or user) R engine, which resides on the desktop.

• This R engine consists of the base R packages, the ORE packages, and any other R packages that the user may have installed.

• At this level, the Transparency Layer intercepts R functions for in-database execution.

• It also enables interactive display of graphical results, while flow control remains with the R environment.

• From the client, users can submit entire R scripts for execution by Oracle Database, using embedded R execution.

• And, although not explicitly depicted here, users can connect to a Hadoop Cluster by using Oracle R Connector for Hadoop.

The second compute engine is Oracle Database.

• This database allows scaling to large data sets.

• R users are able to access tables, views, and external tables, as well as data that is accessible through database links.

• The SQL generator through the Transparency Layer can automatically leverage database parallelism.

• It can also leverage both new and existing in-database statistical and data mining capabilities.

The third compute engine (or engines) are those spawned and managed by Oracle Database, and they execute on the database server machine.

• These embedded R engines enable more efficient data transfer between the database and R.

• Because these engines run on the database server, rather than on the client, they are likely to have greater memory capacity and compute power. Exadata is an example.

• The embedded R execution enables parallel data transfer, returning rich XML or PNG image output, SQL access to R, and the ability to run parallel simulations.

• The embedded R engines also enable use of 3rd party packages or custom functions that do not have in-database SQL equivalent functionality.

• The engines also enable R users to write and test map-reduce scripts before rolling them out to a Hadoop cluster.

• Finally, these engines enable “lights-out” execution of R scripts; that is, scheduling or triggering R script packages inside a SQL or PL/SQL query.

23


The ORE target environment design provides a comprehensive, database-centric

environment for end-to-end analytic processes in R, with immediate deployment to production

environments. It provides many benefits, including:

• Elimination of R client engine memory constraint

• Execution of R scripts through the Oracle Database server machine for scalability and

performance

• Seamless integration of Oracle Database as the HPC environment for R scripts,

providing data parallelism and resource management

• The ability to operationalize entire R scripts in production applications

• Scoring of R models in Oracle Database

24


R and ORE can receive data from many sources. In this figure, we depict the R engine

running on the user’s laptop, as shown in the previous slide.

Through a series of R packages, R itself is able to access data stored in both files, and in

databases.

In addition, ORE provides transparent access to data stored in the local Oracle Database, as

we previously discussed.

In addition, ORE has access to:

• Data in other databases, which are accessible through database links

• Data in external tables

• And, of course, data in HDFS. In addition to bulk import, ORE makes it possible to

access Hadoop directly, in a similar fashion to external tables, by using HDFS connect.

This means that you can join Hadoop data with database data.

25


Here are a few resources on Oracle R related technology.

27


So, in this lesson, we covered three primary topics.

• First, you learned what R is, who uses it, and why they use it.

• Then, we looked at some common user interfaces for R.

• Finally, we discussed Oracle’s strategy for supporting the R community, including an

overview of goals, definitions of software terms, high- and mid-level architecture,

software component features, and R user-community definitions.

29


You’ve just completed “Introducing Oracle R Enterprise”. Please move on to the next lesson

in the series: “Getting Started with ORE”.

30

paced lesson entitled “Introducing Oracle R Enterprise ... · Introducing Oracle R Enterprise 1.4...

Documents

Transcript of paced lesson entitled “Introducing Oracle R Enterprise ... · Introducing Oracle R Enterprise 1.4...