SAS SQL UNDER THE HOOD

Debby Gear DG Consulting

Feb 28, 2020

Make the Most of What you Have

• SAS Proc SQL

• SAS Work

• SQL With Clause

• Why do we Care?

• Supporting Tips

SAS PROC SQL

Proc sql;

create table myTable as

select cust_ID, cust_Name

from myDB.customer;

quit;

Example of SAS WorkSpace

• The directory can be found in the properties of work.

Example of SAS WorkSpace

• Every time SAS EG Opens or a SAS Batch running SAS creates an independent directory and / or a SAS Utility Directory for storing temp files, sorting and other background functions.

• These directories often get huge.

SQL With Clause

• Use the WITH clause to improve query speed for complex subqueries, without the need for conversion. Also known as subquery factoring or Common table expression (CTE), and is used when a subquery is started multiple times.

SQL With Clause

A single with clause can introduce multiple query names by separating them with a comma (the with keyword is not repeated). Each of these queries can refer to the query names previously defined within the same with clause.

SQL:1999 added the with clause to define “statement scoped views”. They are not stored in the database schema: instead, they are only valid in the query they belong to. This makes it possible to improve the structure of a statement without polluting the global namespace. Used by Netezza, Oracle, SQL Server, etc.

SAS PROC SQL uses SAS 92 standards which has not integrated the with clause. When using Pass Through SQL, the standard used is based on your database standards.

Benefits of the With Clause

• Modularize extracts, make the SQL code easier to understand

• Do not need the ability to have write access to Database tables. Especially useful in an environment where Database loading is restricted.

• Improved query performance.

• Minimize data transfer, reduces SAS work/utility datasets.

• Reduces the number of times SAS Admin kills your SAS job or your job fills up SAS work.

Basic Syntax of a With Clause

WITH query_name1 AS (SELECT ...)

, query_name2 AS (SELECT ...

FROM query_name1...

)SELECT ...

WHY DO WE CARE

SAS Pass Through SQL to SAS workproc sql;CONNECT TO Netezza as NZ (server=&NZ_IPaddress. database=BDM_BMO_DM authdomain=&BBR_authD.);create table nccs1 asselect * from connection to NZ(select a.sp_act_key as spactkey

,a.TIME_DIM_KEY as timekey... more stuff here …..

,b.chrgoff_dtfrom basel.bsl_cl_rc_act_f a, basel.BSL_CL_RC_ACT_D_V b where … your join and conditions here ….order by a.sp_act_key); disconnect from NZ;quit;

proc sql;create table limit2 asselect conn_lead_act_sk,sum(b1mago) as install_balfrom nccs1 where spactkey<>conn_lead_act_sk and conn_lead_act_sk <> . and b1mago > 0group by 1;

create table nccs asselect a.*,coalesce(b.install_bal,0) as install_balfrom nccs1 a left outer join limit2 bon a.spactkey=b.conn_lead_act_sk;

proc sql;create table ploc_pop as select * from nccs where spactkey not in (select distinct spactkey from comm)order by spactkey

;quit;

I/O happens at each create

Using With Clause and SASproc sql;CONNECT TO Netezza as NZ (server=&NZ_IPaddress. database=BDM_BMO_DM authdomain=BDMDM);

create table PLOC_results as select * from connection to NZ(/* get ploc accounts */

with ploc_pop as ( select a.sp_act_key…. more stuff here ….. ,b.ln_sec_cd

from basel.bsl_cl_rc_act_f a,basel.BSL_CL_RC_ACT_D_V b

where a … your join and conditions here …. order by a.sp_act_key

), /* comments are always helpful, summarize b1mago by connection */

limit2 as ( select conn_lead_act_sk,sum(b1mago) as install_bal

from ploc_popwhere spactkey<>conn_lead_act_sk and conn_lead_act_sk is not null and b1mago > 0 group by 1

), /* append to results to population */

ploc_pop2 as (select a.* ,coalesce(b.install_bal,0) as install_bal

from ploc_pop a left outer join limit2 b on a.spactkey=b.conn_lead_act_sk

), /* get deliquency history*/

nccs_mago as (select a.sp_act_key as spactkey

,a.time_dim_key/*... more stuff here */ ,b.CR_LIM_AMT

from basel.bsl_cl_rc_act_f a, BASEL.BSL_CL_RC_ACT_D_V b ,basel.time_d c, ploc_pop dwhere a.sp_act_key =b.sp_act_key and a.time_dim_key=c.time_dim_key and a.a.sp_act_key =d.spactkey and c.period_type='MONTH‘

), /* aggregate balances from month ago*/

mago_agg as (

select sp_act_key, sum(ME_CUR_BAL_AMT) as mago_CUR_BAL_AMT, sum(orig_me_cur_bal_amt) as mago_orig_me_cur_bal_amt

from nccs_magogroup by sp_act_key

) /* EMPHASISE NO COMMA ON LAST STEP */

/* FINAL OUTPUT TO CREATE PLOC_results SAS dataset*/select a.*, b.mago_CUR_BAL_AMT, mago_orig_me_cur_bal_amt

from ploc_pop a left join mago_agg b

on a.sp_act_key=b.sp_act_key); disconnect from NZ;quit;

I/O only happens once at the end

What’s the big deal

• Actual client savings• Huge time and I/O savings• Not filling up SAS work space • The more that is done within the sub query saving grow exponentially

SUPPORTING TIPS

How the heck do I find an error

• Cut from the SAS log and paste into your favourite SQL tool, it will provide more comprehensive error messages

• ERROR: CLI prepare error: Unable to retrieve error message.• SQL statement: with ploc_pop as ( select a.sp_act_key as sp_act_key ,a.TIME_DIM_KEY as timekey ,a.ME_CUR_BAL_AMT as b1mago • ,a.nccs_delq_stat_cd as d1mago ,a.ME_CUR_BAL_AMT ,a.ce_bal_amt ,a.orig_me_cur_bal_amt ,b.prod_src_ref_id as prod_id• ,b.cr_lim_amt as crlimit ,b.sp_act_open_dt as opendatx ,b.src_sys_act_no as actsysid ,b.ACT_BL_CD ,b.ME_BLOCK_CODE ,b.MAT_DT• ,b.RECON_DATA_MART_OWNER_CD ,b.RECON_GL_NO ,b.me_cr_lim_amt ,b.appl_dim_key ,b.conn_lead_act_sk ,b.chrgoff_dt• ,b.PREV_ACT_BL_CD ,b.act_prfl_start_dt ,b.ln_sec_cd from basel.bsl_cl_rc_act_f a, basel.BSL_CL_RC_ACT_D_V b where • a.sp_act_key=b.sp_act_key and a.time_dim_key = 37652 and b.act_prfl_end_dt >= '2018-07-31' and b.act_prfl_start_dt <= • '2018-07-01' AND ( substr(b.prod_src_ref_id,1,4) in ('PLOC', 'MCPL', 'IPLO') OR substr(b.prod_src_ref_id,1,5) in ('SBLOC', • 'MCSBL' ,'SBIPL') ) and b.RECON_RESP_NODE not in (SELECT DISTINCT LVL14_RESP_NODE FROM BASEL.BSL_RESP_D WHERE • LVL60_RESP_NODE = 'K9880' AND RESP_NODE_TO_DT >= '2018-07-31' and resp_node_fr_dt <= '2018-07-01' order by LVL14_RESP_NODE) • order by a.sp_act_key ), limit2 as ( select conn_lead_act_sk,sum(b1mago) as install_bal from ploc_pop where • sp_act_key<>conn_lead_act_sk and conn_lead_act_sk is not null and b1mago > 0 group by 1 ), ploc_pop2 as ( select a.* • ,coalesce(b.install_bal,0) as install_bal from ploc_pop a left outer join limit2 b on a.sp_act_key=b.conn_lead_act_sk ), • nccs_mago as ( select a.sp_act_key ,a.time_dim_key ,a.nccs_delq_stat_cd ,a.ME_CUR_BAL_AMT ,a.orig_me_cur_bal_amt• ,a.me_pymt_amt ,b.CR_LIM_AMT ,b.act_prfl_end_dt ,b.act_prfl_start_dt from basel.bsl_cl_rc_act_f a, BASEL.BSL_CL_RC_ACT_D_V b • ,basel.time_d c, ploc_pop2 d where a.sp_act_key =b.sp_act_key and a.time_dim_key=c.time_dim_key and a.time_dim_key between • 37652 -18 and 37652 and c.period_type='MONTH' and a.sp_act_key =d.sp_act_key order by a.sp_act_key, time_dim_key ), • mago_aggregate as ( select sp_act_key, sum(ME_CUR_BAL_AMT), as mago_CUR_BAL_AMT, sum(orig_me_cur_bal_amt) as • mago_orig_me_cur_bal_amt from nccs_mago group by sp_act_key ) select a.*, b.mago_CUR_BAL_AMT, mago_orig_me_cur_bal_amt from • ploc_pop a left join mago_aggregate b on a.sp_act_key=b.sp_act_key.

How the heck do I find an error

• Simply paste into your favourite SQL tool

• Providing an error message that is more detailed(in most cases)

Just Remember inside of passthroughSQL it’s the database Syntax

• Quick Date Tip

• Dates selected from Netezza, Oracle, SQL Server need to be in a format recognized bythe database; single quote and formatted

‘yyyy-mm-dd’ ‘2019-03-12’

• SAS Date format ’12MAR2019’d are notrecognized

Summary

• Save I/O and time, process your data in the database

• And most important • Know your data

• Know your data

• Know your data

Resources

• SAS Communitieshttps://communities.sas.com

• IBM Knowledge Center - Netezza SQL command referencehttps://www.ibm.com/support/knowledgecenter/en/SSULQD_7.2.1/com.ibm.nz.dbu.doc/r_dbuser_ntz_sql_command_reference.html

• Modern SQLhttps://modern-sql.com/feature/with

• Overview of Workspace Servers and Stored Process Servershttps://documentation.sas.com/?docsetId=biasag&docsetTarget=n03001intelplatform00srvradm.htm&docsetVersion=9.4&locale=en

QUESTIONS?