What´s New? SAP HANA SPS 07 - blog.scherbinek.de´s-New... · What´s New? SAP HANA SPS 07 SAP...
-
Upload
truongtram -
Category
Documents
-
view
238 -
download
0
Transcript of What´s New? SAP HANA SPS 07 - blog.scherbinek.de´s-New... · What´s New? SAP HANA SPS 07 SAP...
What´s New? SAP HANA SPS 07 SAP HANA Application Function Library (AFL) (Delta from SPS 06 to SPS 07)
SAP HANA Product Management November, 2013
© 2013 SAP AG. All rights reserved. 2 Public
Contents
Overview
What’s New for SPS 07: Predictive Analysis Library
What’s New for SPS 07: Application Function Modeler
© 2013 SAP AG. All rights reserved. 4 Public
Application Function Library: General Overview – 1-
Application Function Library (AFL) – what is it?
Pre-delivered commonly utilized business, predictive and other types of algorithms for use in projects
or solutions that run on SAP HANA. The technology framework enabling the use of these algorithms
is called the Application Function Library .
What are it’s benefits?
These algorithms can be leveraged directly in development projects, speeding up projects by
avoiding writing custom complex algorithms from scratch. AFL operations also offer very fast
performance, as AFL functions run in the core of SAP HANA in-memory DB.
What are its primary libraries?
The Predictive Analysis Library (PAL) is a set of functions in the AFL. It contains pre-built, parameter-
driven, commonly used algorithms primarily related to predictive analysis and data mining.
The Business Function Library (BFL) is a set of functions in the AFL. It contains pre-built, parameter-
driven, commonly used algorithms primarily related to finance.
© 2013 SAP AG. All rights reserved. 5 Public
The Application Function Library Framework in SAP HANA
Application Functions (C++)
SQLScript
HANA Clients (App Server, Analytics Technology, etc)
SAP HANA
Business Function Library
Predictive Analysis Library
AFL Framework
…
AFL Technology includes
Application Functions
Written in C++ and delivered as AFL content by SAP
Predictive Analysis and Business Function Library have been
released in SPS 05 as AFL content
AFL Framework
On demand library loading framework for registered and supported
libraries
AFL are consumed for use from SqlScript via so-called wrapper-
procedures. Consumption can be controlled via permissions.
Beyond the initial script-based approach, the Application
Function Modeler is released with SPS 06 as a graphical editor
to facilitate the design-time process of creating the
wrapper-procedures and can easily be re-used as part of
development workflow.
Parameter Table
AFM
© 2013 SAP AG. All rights reserved. 6 Public
Predictive Analysis Library – General Overview
Predictive Analysis Library (PAL)
• Compiled analytic function library for predictive analysis in HANA SPS 05
• Support multiple algorithms: K-Means, Association Analysis, C4.5 Decision Tree, Multiple
Linear Regression, Exponential Smoothing…
Know Your Business
Decide with Confidence
Compute Quickly
Uncover deep insights & patterns about the business: association rules,
customer clustering, or sales prediction
Drive more advanced analyses. Decision is made with support from
analysis numbers
Query and analyze data in real-time with high-performance
computation in-memory
Help Customers To
Bring decision support capabilities to the business users through
simplified experience and pre-built scenarios Empower the business
© 2013 SAP AG. All rights reserved. 7 Public
The Predictive Analysis Library in SAP HANA Functional Overview as of SPS 06
The Predictive Analysis Library (PAL) is a built-in C++ library to perform in-database data mining
and statistical calculations, designed to provide excellent performance on large data sets.
Embedded in SAP HANA via the Application Function Library
Data Preparation
Sampling
Binning
Scaling
Convert Categorical to Binary
Outlier Detection
Inter-Quartile Range Test (Tukey’s Test)
Variance Test
Anomaly Detection
Cluster Analysis
ABC Classification
DBSCAN
K-Means
Kohonen Self Organized Maps
Time Series Analysis
Single Exponential Smoothing
Double Exponential Smoothing
Triple Exponential Smoothing
Link Prediction
Different Measurements: Common
Neighbors; Jaccard’s Coefficient;
Adamic/Adar; Katzβ
Other
Weighted Scores Table
Classification Analysis
C4.5 Decision Tree Analysis
CHAID Decision Tree Analysis
K Nearest Neighbour
Multiple Linear Regression
Polynomial Regression
Exponential Regression
Bi-Variate Geometric Regression
Bi-Variate Logarithmic Regression
Logistic Regression
Naïve Bayes
Association Analysis
Apriori
Apriori Lite
Status: SAP HANA SPS 06.
Y
X Z
© 2013 SAP AG. All rights reserved. 8 Public
Application Function Modeler
Application Function Modeler A graphical editor to facilitate a faster and easier design-time process of creating the wrapper-procedures
AFL Models are stored as repository objects and can easily be re-used as part of development workflow.
Library Selection
Function List and
Search
Parameters and
specifications
for table types
Model Editor
Drag‘n drop of functions
Template for table types
Data source selection and
automatic mappings to table
types
Sample SQL for
procedure
consumption
© 2013 SAP AG. All rights reserved. 10 Public
New Algorithms: Statistical Functions
Univariate
Mean, median, variance, standard deviation
Kurtosis
Skewness
Multivariate
Covariance matrix
Pearson correlations coefficient matrix
Chi-squared tests:
– test of quality of fit
– test of independence
Variance equal test (F-test)
© 2013 SAP AG. All rights reserved. 11 Public
New Algorithms: Substitute Missing Values; Partitioning
Substitute missing values
Continuous variable: replace the ‘null’ value with ‘mean’ or ‘median’
Categorical variable: replace the ‘null’ value with ‘mode’
Partitioning Splits an input table into 3 parts (training table, testing table and validation table).
Union could be a subset of the input table and the validation part could be empty
Two partition methods are supported:
– Random Partition
– Stratified Partition (stratified sampling)
© 2013 SAP AG. All rights reserved. 12 Public
New Algorithms: Support Vector Machine; Forecast Smoothing
Support Vector Machine
A family of supervised machine learning algorithms that analyze data and
recognize patterns, typically used for classification and regression analysis
Holds advantages over other supervised learning models: SVM models can be
either linear or non-linear
Forecast Smoothing
Calculates a set of optimal parameters for PAL functions: Single Smoothing,
Double Smoothing and Triple Smoothing
Outputs the forecasting result by using the best parameter
Parameter optimization is based on global and local search algorithms
© 2013 SAP AG. All rights reserved. 13 Public
New Clustering Algorithms: Agglomerate Hierarchical; Affinity Propagation
Agglomerate Hierarchical Clustering
Widely used clustering method which can find natural groups within a set of data
Groups the data into a hierarchy or a binary tree of the subgroups
The PAL version follows the agglomerate approach: merges the clusters with a
bottom-up strategy.
Affinity Propagation Clustering
A relatively new clustering algorithm based on the concept of "message passing"
between data points
Holds the advantage that the number of clusters does not necessarily have to be
predetermined
© 2013 SAP AG. All rights reserved. 14 Public
Enhancements to Regression Algorithms
Linear regression
Support p-value for each coefficient
Stepwise linear regression (forward and backward only)
Logistic regression
Logistic regression with L1 regularization
Categorical support for input variable
p-value for coefficient
© 2013 SAP AG. All rights reserved. 15 Public
Enhancements to Other Algorithms
Apriori
Output rules in relational format
Add lift as a filter for rules
Provide an option which controls the number of items in right hand side (1 or many)
Decision Tree
Missing value handling in decision tree (mean/mode)
© 2013 SAP AG. All rights reserved. 17 Public
Application Function Modeler SPS 07 Features Overview
Drive AFM achieve a more stable and usable solution
Better integrate with AFL Framework.
UI Improvement: more friendly and usable
Better support the E2E Development Scenarios: create, activation, calling, delivery etc.
Others: Performance improvement, Error Handling
Objectives
Performance improvement: creation, model consistence
check
Error Message: in studio side and server side
Support open file offline but read-only
…
Other
Better support the table type editing
Better naming rule and icon
Better UI Text
UI Improvement
Update table type constrains and default parameter
Support new function in PAL (11 new functions)
Use new AFL API to generate wrapper procedure
Refactor the data model
AFL Framework
View support
Better undo/redo
Calling parameter remove/add/restore/persistence
Schema mapping
…
E2E Development Scenarios
© 2013 SAP AG. All rights reserved. 18 Public
AFM: New Functions
New Algorithm in AFM in SPS 07 Description Category AFM PAL
DBSCAN DBSCAN Clustering SPS 07 SPS 06
LINKPREDICTION Link Prediction Social network analysis SPS 07 SPS 06
CONV2BINARYVECTOR Convert category Type to
Binary Vector Preprocessing SPS 07 SPS 06
SLIGHTSILHOUETTE Slight Silhouette Clustering SPS 07 SPS 06
NBCTRAIN NBCTRAIN Classification SPS 07 SPS 06
NBCPREDICT NBCPREDICT Classification SPS 07 SPS 06
HCAGGLOMERATE Hierarchical Clustering Clustering SPS 07 SPS 07
SVMTRAIN Train with Support Vector
Machines Classification SPS 07 SPS 07
SVMPREDICT Predict with Support Vector
Machines Classification SPS 07 SPS 07
PARTITION Data Partition Preprocessing SPS 07 SPS 07
SUBSTITUTE_MISS Substitute Missing Values Preprocessing SPS 07 SPS 07
….
11 new functions will be supported
© 2013 SAP AG. All rights reserved. 19 Public
AFM: Updated Generated Procedure Definition
New AFL Generator API in Runtime Plugin SPS 06: CREATE PROCEDURE ... LANGUAGE LLANG AS BEGIN ...
SPS 07: CREATE PROCEDURE ... LANGUAGE AFLLANG
SQL SECURITY INVOKER READS SQL DATA AS BEGIN ..
© 2013 SAP AG. All rights reserved. 20 Public
AFM: Table Type Editor & Parameter Editing
Table Type Editor
Parameter Editing
Support user add/remove parameter
Provide restore defaults action
© 2013 SAP AG. All rights reserved. 21 Public
AFM: New Error Messages
Provide better error messages and provide the possible next step
Case Error Message (Client Side)
No required permission User does not have the AFL__SYS_AFL_AFLBFL_EXECUTE role. Assign the role first.
Table Type Configuration Error Table column setting does not match function specification
Calling with activation AFL Model (.aflpmml) file has not been activated. Activate the file before calling it.
… …
(Server side Activation)
No schema sleeted Repository: Error in repository runtime extension. No schema selected. Select a schema.
No Function Selected Repository: Error in repository runtime extension. No function in the model. Select a function.
Table Type has not been initialized Repository: Error in repository runtime extension. Add columns to the Dataset table.
File corrupt Repository: Error in repository runtime extension. Invalid file. Check the file or create the file again.
… …
© 2013 SAP AG. All rights reserved. 22 Public
Disclaimer
This presentation outlines our general product direction and should not be relied on in making
a purchase decision. This presentation is not subject to your license agreement or any other
agreement with SAP.
SAP has no obligation to pursue any course of business outlined in this presentation or to
develop or release any functionality mentioned in this presentation. This presentation and
SAP’s strategy and possible future developments are subject to change and may be changed
by SAP at any time for any reason without notice.
This document is provided without a warranty of any kind, either express or implied, including
but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or
non-infringement. SAP assumes no responsibility for errors or omissions in this document,
except if such damages were caused by SAP intentionally or grossly negligent.
Thank you
Contact information
Ron Silberstein
SAP HANA Product Management
To get the best overview of what’s new in SAP HANA SPS 07, read this blog.
© 2013 SAP AG. All rights reserved. 24 Public
© 2013 SAP AG. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG.
The information contained herein may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and
SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in
the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other
countries.
Please see http://www.sap.com/corporate-en/legal/copyright/index.epx#trademark for additional trademark information and notices.
© 2013 SAP AG. All rights reserved. 25 Public
© 2013 SAP AG. Alle Rechte vorbehalten.
Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche
Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.
Einige der von der SAP AG und ihren Distributoren vermarkteten Softwareprodukte enthalten proprietäre Softwarekomponenten anderer Softwareanbieter.
Produkte können länderspezifische Unterschiede aufweisen.
Die vorliegenden Unterlagen werden von der SAP AG und ihren Konzernunternehmen („SAP-Konzern“) bereitgestellt und dienen ausschließlich zu Informationszwecken.
Der SAP-Konzern übernimmt keinerlei Haftung oder Gewährleistung für Fehler oder Unvollständigkeiten in dieser Publikation. Der SAP-Konzern steht lediglich für Produkte
und Dienstleistungen nach der Maßgabe ein, die in der Vereinbarung über die jeweiligen Produkte und Dienstleistungen ausdrücklich geregelt ist. Keine der hierin
enthaltenen Informationen ist als zusätzliche Garantie zu interpretieren.
SAP und andere in diesem Dokument erwähnte Produkte und Dienstleistungen von SAP sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP
AG in Deutschland und verschiedenen anderen Ländern weltweit. Weitere Hinweise und Informationen zum Markenrecht finden Sie unter http://www.sap.com/corporate-
en/legal/copyright/index.epx#trademark.