SAS94 and SAS Viya code Some differences · SAS Viya and programming differences from SAS94...

Company Conf ident ia l – For Internal Use OnlyCopyright © SAS Inst itute Inc. A l l r ights reserved.

SAS94 and SAS Viya code Some differencesTor Ove Kilnes

SAS Institute

Trondheim and Tromsø 18-19 September 2019


AgendaSAS Viya and programming differences from SAS94

• Fundamental architecture of SAS Viya - SAS94 and CAS together

• Load of data – Principles of caslibs

• Examples of differences in code – CAS Viya enabled SAS94 code

• What procedures and code do not run in CAS

- Examples like proc sql and proc freq

• Sorting of data, by-processing and threads

• CAS actions and actions-sets – Proc CAS Language

• Brief process for migrating VA7.x to VA8.x (Viya)


3

Fundamental architecture - SAS94 and CAS togetherWhy run in CAS?


Why use SAS Viya programming?Multi-threaded runtime and new analytical procedures

Analytical Innovation

Performance

B

• CAS is a runtime environment in SAS® Viya™ for data management and analytics in SAS

• CAS is a platform for high-performance distributed computing and analytics

• CAS is designed to work on a single server or in a clustered environment

• CAS is designed for the Cloud (for example Amazon Web Services or Azure) but can also be hosted on premise.


Improvement examples running in CAS over SAS94

Financial Use Case - Monte Carlo Simulations for Delinquency DATA Step

~20,000 lines CPU intensive codeSource table ~1TBTarget table ~50GB

26 Iterations of DATA Step SAS Workspace Server - 26 hours total.

Performance gainSAS CAS Server – 2.5 hours total.

Government Use Case – Analytical Base Table creation for Modeling and ScoringDATA Step and Base SAS Procedures

Source tables: ~220GB – 230GBTarget tables: ~44GB – 46GB

SAS Workspace Server - 56 hours total.

Performance gainSAS CAS Server – 9 hours total.

B


The SAS Platform

SAS 9.4

SAS Enterprise Guide

SAS Studio

SAS DataIntegration

Studio

SAS Studio for Viya

SAS Viya

SAS and external programming interfaces

B

External interfaces

SAS interfaces

I/O and disk based

processing

In-Memory based

processing


SAS Viya platform has an embedded SAS94 engineSPRE

SAS Viya - CAS

SAS CAS ServerSAS Workspace Server

Single Threaded→

Regular performance

as SAS94

Multi- threaded→

Improved performance for

Viya enabled code

BB

SPRE (SAS Programming Runtime Environment)


Distribute load on multiple processing cores in ViyaOne or more machines, multi-thread

C

One processing core

SAS94 sessionSAS Viya session

Multiple Processing cores

in parallell


SAS Viya things to remember:Multi-threading are not always a friend..

• Datastep summary with statements like “sum_i + i;” in multithread mode, gives the sum of each thread and not the sum of the total table.

• To avoid this, run single-threaded using datastep option single=yes

• Usage of DIF, LAG, _N_, First.Var, Last.Var, Retain etc. should be run single threaded, as it need information about the previous row.

C


SAS Viya things to remember:

• Existing SAS94 code can run as-is in Viya on SPRE• This does not benefit from in-memory processing

• You can adapt the SAS94 code to become Viya/CAS enabled code.• This will benefit from in-memory processing • The performance gains may be large (… but not always possible…)

B


Examples of differences in proceduresSome are Viya enabled, some are not

• The list of Foundation procedures (e.g. only Visual Analytics licensed) is at: https://go.documentation.sas.com/?docsetId=pgmdiff&docsetTarget=p1e2swghnyju6fn13jr3935jpsaw.htm&docsetVersion=3.4&locale=en

• Some of these are marked «YES» for CAS-processing

• There are also new Foundation procedures that is Viya-enabled, like proc MDSummary• There are also new procedures in other Viya-modules (like SAS Visual Statistics)

• Examples that is not Viya enabled is:• Proc Sort

• Proc Sgplot

• Proc SQL

• Some procedures is “in-between”, like proc Transpose, if data- and out-options points to CAS libnames

https://go.documentation.sas.com/?docsetId=pgmdiff&docsetTarget=p1e2swghnyju6fn13jr3935jpsaw.htm&docsetVersion=3.4&locale=en


Where is data processed?Watch out for messages in the log..

SAS Viya

BB

SPRE (SAS Programming Runtime Environment)

• If target data is a SAS94 libname

• If source data is a SAS94 libname

• If source and target data is in SAS94

• If the proc used not supports Viya (is not Viya enabled)

• Essentially if source and target is a CASLIB

• The proc is Viya enabled.

• If the data is in a CASLIB and the proc not is Viya enabled, data will be pushed between SAS94 and Viya

• Similar if source and target is SAS94 and a caslib.

• Such data movement is not efficient.

• Some SAS94 proc support caslib as source and target (proc freq), while others do not (proc sql)

• Look for info in the log !!

Runs in SAS94

Runs in CAS


13

Load of data – Principles of caslibs and sessions


What is a caslib?

...

Caslib

Data source

In-Memory private session scope

SASHDAT

CAS tables

An In-memory table that is loaded from a source

C

• A Caslib is similar to a libname in SAS94, but it points to an in-memory table

• The source of the data can be any source that can be accessed using SAS94 code or directly in Viya enabled access engines

• The in-memory session is either a “session scope” (e.g. private) or “global scope (e.g. shared with others

In-Memory global session scope


Private Session “scope” or Global session “Scope”?Load tables for private usage or shared usage

Just visible and available to you in your session

When the CAS session terminates, the data is

DELETED

Session Scope

Promote=No

Global Scope

Visible across CAS sessions to all that is authorized for the caslib.

When the CAS session terminates, the data still AVAILABLE

Promote=YES

C


Readyness toolSAS PSD has a tool scanning through your programs

• SAS PSD has a tool that scans and checks if your old SAS94 programs is Viya enabled and reports what part of the code that contains items not supported by CAS.

• Keywords:

• LOC (Lines of Code)

• Steps (Number of code steps)

• CAS (Code Not supported Viya code)

• Macro (number of macros)

• Include (Number of include statements, not evaluated code)

• Example here is the test code used in examples in this presentation


CAS Language – Proc CAS

• Proc CAS is a SAS procedure for sending scripting language (CASL, CAS Language) that prepares arguments for execution of CAS actions using the CAS server.

• CASL can also be used from external languages like Python, Lua or R

• The actions is what is being done, and organized in action sets for functionality types, like Aggregation, Machine Learning or Table.


19

Code examples


Create a CAS sessionExample with various options

** SAS Viya test code..;

** General options. For extra loginfo (msglevel) and use of parallell processing (any or None) when possible..;

options dsaccel=any msglevel=i ;

options fullstimer;

options set=CASCLIENTDEBUG=1;

** UTF8: Conversion of special characters automatically, using option CASNCHARMULTIPLIER ;

** For Double Byte Character sets, use value 2.0 ;;

options CASNCHARMULTIPLIER=1.5;

** Option for forcing usage of _authinfo file..;

*options set=CAS_AUTH_METHOD=authinfo;

* This file should be named .authinfo and have permissions 600 in users home folder..;

*options authinfo='/home/<userid>/.authinfo';

** This will create session "mySess", and add extra time metric info on CAS processing ;

options sessref='mySess' ;

cas mySess sessopts=(caslib=casuser timeout=1800 locale="en_US" METRICS=true);

caslib _ALL_ assign;

** List CAS sessions..;;

cas mySess listsessions


Load data from SAS94 and promote to globalExample with datastep and proc casutil

** Make a casuser table out of sashelp.cars..;

data casuser.cars;

set sashelp.cars;

run;

data casuser.cars2;

set casuser.cars;

run;

** Delete promoted castable in caslib PUBLIC if it exists before promotion.. ;;

proc casutil;

droptable incaslib=public casdata="cars" quiet;

droptable incaslib=public casdata="cars2" quiet;

run; quit;

** Promoter datasett og vis content..;;

proc casutil incaslib=casuser outcaslib="public";

promote casdata="cars";

contents incaslib=public casdata="cars";

quit;

NOTE: The promotion may remove casuser.cars

Rerun of step may give error: Casuser.cars do not

existReason is to save memory


Data processing in Viya or SAS94? (1/2)Example with proc SQL/FedSQL in SAS94

options msglevel=i;

** create WORK.test as testdata in SAS94..;;

data WORK.test;

*do i = 1 to 1000000000; * THis takes too long time..;;

do i = 1 to 10000;

j=1;

output;

end;

run;

* Run fedSQL in SAS94..;;

proc fedsql;

select sum(j) as sumJ

from WORK.test ;

quit;


Data processing in Viya or SAS94? (2/2)Example with proc SQL/FedSQL in CAS

** create CASUSER.test as testdata in CAS..;;

data casuser.test;

*do i = 1 to 1000000000;

do i = 1 to 10000;

j=1;

output;

end;

run;

** Run in CAS using casuser as caslib. NOTE the reference to the CAS session "mySess" ;

proc fedsql sessref=mySess;


from casuser.test ;

quit;

* Due to proc sql (and not fedsql), SAS tries to "dump" data from CAS to SAS94 ;;

* In this case data is pushed between CAS and SAS94 implicitly..;;

proc sql;

create table work.aa as


from CASUSER.test

;

quit;

* In this example data is loaded from caslib casuser, processed in SAS94 and saved in caslib casuser..;;

proc freq data=casuser.test;

table j / out=casuser.sumtest_freq;

run;

NOTE: Sometime reference to the CAS Session is required

NOTE: Proc Freq is not Viya Enabled, so data is processed in SAS94, but both source and

target Is in CAS


Data processing with threadsExample with summary of data* Create test data i SAS94 for summary ;

data testthreads;

do i = 1 to 100000;

j=1;

output;

end;

run;

* Run code in SAS94 - Result is 1 row ;

data testthreadssas(drop=i j);

retain sum_i sum_j;

set testthreads end=eof;

sum_i + i;

sum_j + j;

if eof;

run;

* Create test data i CAS, using default private session caslib CASUSER ;

data casuser.testthreads;

do i = 1 to 100000;

j=1;

output;

end;

run;

* When run in CAS - gives more than 1 row ;

* as it has multiple threads which in total is the total sum.. ;

data casuser.testthreadscas(drop=i j);

retain sum_i sum_j;

set casuser.testthreads end=eof;

sum_i + i;

sum_j + j;

put _threadid_= _nthreads_=;

if eof;

run;

** SOLUTION: Run datastep in a single thread..Gives 1 row as in SAS94 ;

data casuser.testthreadscas1(drop=i j) / single=yes;

retain sum_i sum_j;

set casuser.testthreads end=eof;

sum_i + i;

sum_j + j;

put _threadid_=;

if eof;

run;


Data processing with by-processing and sortExample avoiding proc Sort in by-group summary

** Sum a variable for each BY group *;

** SAS94 *;

** In SAS94 we need to sort table and then we can use BY statement in DATA step *;

proc sort data=sashelp.class out=work.class;

by sex;

run;

data work.class_sum_by_sex;

set work.class;

by sex;

if first.sex then weight_sum = 0;

* Sum weight for each sex *;

Sex_weight_sum + weight;

if last.sex then output;

keep sex Sex_weight_sum;

run;

title 'DATA Step in SAS';

proc print data=work.class_sum_by_sex;

run;

** CAS *;

** First load sashelp.class to casuser..;;

data casuser.class;

set sashelp.class;

run;

* In CAS we do not need to sort table and then we can use BY statement in DATA step *;

data casuser.class_sum_by_sex;

set casuser.class;

by sex;

if first.sex then weight_sum = 0;

* Sum weight for each sex *;

Sex_weight_sum + weight;

if last.sex then output;

keep sex Sex_weight_sum;

run;

title 'DATA Step in CAS';

proc print data=casuser.class_sum_by_sex;

run;

Same results, but no sorting in CASThe only change is caslib instead of libnames


Load and save data to disk in ViyaExample with .sashdat files

** Allocate caslib tok;

caslib tok datasource=(srctype="path") path="/data" ; ** global NOTACTIVE;

caslib _ALL_ assign; ** Show in canvas..;;

** Un-load the table from caslib tok in case it exists..;

proc casutil ;

droptable incaslib=tok casdata="class" quiet;

run;

** Loading the sashelp.class table into caslib tok and saves to disk. Lists content on disk..;

proc casutil ;

load data=sashelp.class outcaslib=tok ; ** Load without promotion to outcaslib tok..;;

contents incaslib=tok casdata="class"; * Show contents of castable tok.class ;

save incaslib=tok casdata="class" replace ; ** Save data tok.class to disk, in default format .sashdat ;;

list tables incaslib=tok; ** Lists the current tables in caslib tok..;;

list files incaslib=tok; ** Lists the current files in caslib tok, like sashdata types..;;

run;

** Dropping CAS table class from memory and reload from disk (.sashdat file)..;

proc casutil ;

droptable incaslib=tok casdata="class" quiet;

run;

** Using proc casutil to reload the saved castable tok.class from disk ..;;

** NOTE: The name of the file to load need its file extension, but its PATH is from the caslib referenced ;;

proc casutil ;

load outcaslib=tok casout="class" casdata="class.sashdat" importoptions=(filetype="hdat") ; ** Reading from disk, the table class in default format .sashdat ;;

run;

** Un-allocate caslib tok..;

caslib tok drop;

From statement:list files incaslib=tok;


Use of SAS macros and if Vs where statementExample macro and subsetting with if or where statement%let whr=F;

options nofullstimer ; ** Turning this off to get cleaner log..;;

** NOTE: Here sashelp.class is loaded into CAS using SPRE. Then the macrovar is used towards CAS...;;

data casuser.class;

set sashelp.class;

run;

** NOTE: As data is read from SAS94 libname This runs in SPRE..;; ** %macro myMac1;

data class;

set sashelp.class;

* where sex="&whr." ;

if sex="&whr." ;

x=1;

put _hostname_ 'thread #' _threadid_;

run;

%mend myMac1;

%myMac1;

** NOTE: Here data is read and written to CAS. But it is a difference using where or if for subsetting..;;

** Only if subsetting is supported in datastep CAS..;

%macro myMac1;

data casuser.class2;

set casuser.class;

* where sex="&whr.";

if sex="&whr.";

x=1;

put _hostname_ 'thread #' _threadid_;

run;

%mend myMac1;

%myMac1;

Running in SPREBoth Where and If subsetting works

Running in CASOnly If subsetting worksOtherwise it is executed

in SPRE


Usage of proc CAS for summary of a tableExample with proc MDSummary and aggregate action

** proc mdSummary, a new SAS Viya procedure..;;

proc mdsummary data=casuser.prdsale ;

var actual;

groupby country division ; * month ;

output out=casuser.actual_mdsum ;

run;

** CAS Aggregate using CAS action AGGREGATE;;

proc cas ;

session mySess;

aggregation.aggregate result=r status=s /

table={ caslib="casuser",

name="prdsale",

groupBy={"country", "division"},

vars={"actual"} },

varSpecs={

{name='actual', summarySubset={'SUM'}, columnNames={'actual'}}

}

casout={caslib="casuser", name="actual_sum_casagg", replace=True, replication=0} ;

run;

print r; print s;

run; quit ;


Brief principle for migrating VA7.x to VA8.x (Viya)Example macro and subsetting with if or where statement

• For migrating VA7.x reports to VA8.x, the main steps are:

• Export content in SASMC

• Download exported content

• Upload to Viya environment

• Map content between v7 and v8

• Import using command line tool

• Note some difficult objects:

• STP (stays essentially in SAS9)

• Change in users & permissions

Read blog number 2 at: https://communities.sas.com/t5/tag/SAS%20Visual%20Analytics%20Promotion/tg-p/board-id/library

https://communities.sas.com/t5/tag/SAS Visual Analytics Promotion/tg-p/board-id/library


Useful links

• SAS Viya programming documentation

• https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.4&docsetId=mlov&docsetTarget=home.htm&locale=en

• SAS Community SAS9 to Viya (Phil Weiss)

• http://www.sascommunity.org/planet/blog/category/sas-9-to-sas-viya/

• Direct link to SGF 2018 paper: https://blogs.sas.com/content/sgf/files/2018/01/TechnicalWhitePaper_SASViya_Jan2018_FINAL.pdf

• Lex Jansen on new (2017) SAS datastep programming in Viya

• https://www.lexjansen.com/mwsug/2017/BB/MWSUG-2017-BB129-SAS.pdf

• Documentation links

• https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.4&docsetId=pgmsashome&docsetTarget=home.htm&locale=no

• Proc CAS, CAS Actions starter

• https://go.documentation.sas.com/?docsetId=pgmdiff&docsetTarget=p06ibhzb2bklaon1a86ili3wpil9.htm&docsetVersion=3.4&locale=en#n0vrm865vfx4xfn1pm2nifk4l05g

• Promotion of VA7.x to VA8.x (Viya)

• https://communities.sas.com/t5/tag/SAS%20Visual%20Analytics%20Promotion/tg-p/board-id/library

• https://go.documentation.sas.com/?docsetId=calpromotion&docsetTarget=n0ihk3dsgoiqvtn15oce7kguif1s.htm&docsetVersion=3.3&locale=en#n0mmz3x6euvag8n19uhdva0917u1

https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.4&docsetId=mlov&docsetTarget=home.htm&locale=en

http://www.sascommunity.org/planet/blog/category/sas-9-to-sas-viya/

https://blogs.sas.com/content/sgf/files/2018/01/TechnicalWhitePaper_SASViya_Jan2018_FINAL.pdf

https://www.lexjansen.com/mwsug/2017/BB/MWSUG-2017-BB129-SAS.pdf

https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.4&docsetId=pgmsashome&docsetTarget=home.htm&locale=no

https://go.documentation.sas.com/?docsetId=pgmdiff&docsetTarget=p06ibhzb2bklaon1a86ili3wpil9.htm&docsetVersion=3.4&locale=en#n0vrm865vfx4xfn1pm2nifk4l05g

https://communities.sas.com/t5/tag/SAS Visual Analytics Promotion/tg-p/board-id/library

https://go.documentation.sas.com/?docsetId=calpromotion&docsetTarget=n0ihk3dsgoiqvtn15oce7kguif1s.htm&docsetVersion=3.3&locale=en#n0mmz3x6euvag8n19uhdva0917u1


Useful links

• https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2018/2622-2018.pdf

• https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2018/2622-2018.pdf

• https://support.sas.com/resources/papers/proceedings17/SAS0309-2017.pdf

• https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.4&docsetId=casdspgm&docsetTarget=p0arqv3vknobbjn1t58xg8kf85w3.htm&locale=da

• https://go.documentation.sas.com/?activeCdc=pgmsascdc&cdcId=sasstudiocdc&cdcVersion=3.8&docsetId=casref&docsetTarget=p1d4wnstovanu0n1ang5hp3lj9rh.htm&locale=en

• https://go.documentation.sas.com/?cdcId=pgmcdc&cdcVersion=8.11&docsetId=pgmdiff&docsetTarget=titlepage.htm&locale=da

• http://smportal.sas.com/portalcontent/platform/sas-94-sas-viya-functional-comparison.pdf

• https://communities.sas.com/t5/Ask-the-Expert/Leveraging-SAS-Viya-to-Improve-Base-SAS-Processes/ta-p/474181

• https://blogs.sas.com/content/sgf/files/2018/01/TechnicalWhitePaper_SASViya_Jan2018_FINAL.pdf

• https://go.documentation.sas.com/?docsetId=casref&docsetTarget=p0om95zwv9qq78n1teezb9tjl29a.htm&docsetVersion=3.4&locale=en

B

https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2018/2622-2018.pdf

https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2018/2622-2018.pdf

https://support.sas.com/resources/papers/proceedings17/SAS0309-2017.pdf

https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.4&docsetId=casdspgm&docsetTarget=p0arqv3vknobbjn1t58xg8kf85w3.htm&locale=da

https://go.documentation.sas.com/?activeCdc=pgmsascdc&cdcId=sasstudiocdc&cdcVersion=3.8&docsetId=casref&docsetTarget=p1d4wnstovanu0n1ang5hp3lj9rh.htm&locale=en

https://go.documentation.sas.com/?cdcId=pgmcdc&cdcVersion=8.11&docsetId=pgmdiff&docsetTarget=titlepage.htm&locale=da

http://smportal.sas.com/portalcontent/platform/sas-94-sas-viya-functional-comparison.pdf

https://communities.sas.com/t5/Ask-the-Expert/Leveraging-SAS-Viya-to-Improve-Base-SAS-Processes/ta-p/474181

https://blogs.sas.com/content/sgf/files/2018/01/TechnicalWhitePaper_SASViya_Jan2018_FINAL.pdf

https://go.documentation.sas.com/?docsetId=casref&docsetTarget=p0om95zwv9qq78n1teezb9tjl29a.htm&docsetVersion=3.4&locale=en

SAS94 and SAS Viya code Some differences · SAS Viya and programming differences from SAS94...

Documents

Transcript of SAS94 and SAS Viya code Some differences · SAS Viya and programming differences from SAS94...