SAS Datastep and Program Data Vector Present

8/11/2019 SAS Datastep and Program Data Vector Present

1/117

Understanding the SAS DATA Step

and the Program Data Vector

Steven J. First, President

2997 Yarmouth Greenway Drive, Madison, WI 53711

Understanding the SAS DATA Step and the Program Data Vector 1

Phone: (608) 278-9964, Web: www.sys-seminar.com


2/117

Understanding the SAS DATA Step and the PDV

, .

SSC specializes SAS software and offers SAS:

Consulting Services Help Desk Plans

.

SAS is a registered trademark of SAS Institute Inc. in the USA and othercountries. The Missing Semicolon is a trademark of Systems Seminar


Consultants, Inc.


3/117

Global Warming Part 1



4/117

Abstract

e sys em s ma e up o s an e ep

The DATA Step has:

an excellent, full fledged programming language

statements to read and write almost any type of data value

Conversion and calculation of new data

Looping

Interfaces

Arra s

Much, much more.

,

statements, is what makes the SAS language so popular.



5/117

Abstract (continued)

s paper w a ress:

How the DATA step fits with the rest of the SAS System

DATA step assumptions and defaults

internal structures such as buffers, and the Program Data Vector

compiler statements

executable statements



6/117

Introduction

e sys em s or g ns are n e s an s w :

James Goodnight

John Sall

A. J. Barr

others

Concepts in the design include:

self defining files

a s stem of default assum tions

procedures for commonly used routines

data handling step that would evolve into the SAS DATA step



7/117

Introduction

e s ep n my op n on:

extremely simple

elegant design

continues today

30 years of enhancements.



8/117

Structure of SAS

cons s s o :

1. a data handling language (DATA step)

2. a library of pre-written procedures ( PROC step)

S A S S A SSTATEMENTS SUPERVISOR

RAW

DATA

SAS

DATA REPORT

SASDATA

SAS

PROC



9/117

Purpose of the DATA Step

e e a a n s ape or a er s an s eps.

SAS PROCs can only read SAS datasets

We might have some other type of file to process

SAS dataset has built in descriptor that keeps track of names and

attributes

later steps dont have to remember as many details

If we dont have well defined data

DATA step gives us the power to read and write virtually any kind of file

We can do calculations and computations on a single row of data

It has a very powerful data handling language



10/117

SAS DATA Step Overview

s eps can rea an wr e mos ypes o a a s ore on your compu er.

otherSAS

instreamrawdisk,

DBMS,program

ataset tape pro ucts

SAS/ACCESS

get the data inshape for analysis

DBMS,program

rawdisk,

instreamdata

otherSAS

SAS/ACCESS

reports

Notes:


.

Access to non-SAS database management systems requires a

SAS/ACCESS product.


11/117

Default Assumptions

any assump ons are ma e o save me an e or .

As computer scientist I study and use many programming languages

Early on I was intrigued by the cleverness and common sense of SAS

Many tedious programming tasks were eliminated through defaults

System still provided a means to override those defaults when necessary

Our job as a DATA step programmer then is different from other

languages

In many ways our tasks are:

understanding the defaults

knowin how to work with them

override them as necessary.



12/117


13/117

Default Assumptions (continued)

xamp es e au s:

Automatically defining storage areas for each variable referenced without

nee o pre e ne em

A default length of 8 was assumed for all variables

A assumption that a variable is numeric if not specified

LIST input assumed that data values would be separated by blanks rather

than specifying exact columns

SUBSETTING IF statements which imply to continue processing if acon on s rue, e se e e e e o serva on. x. ra e > ;

When no comparison is made in an IF statement, assume to be checking

for 1 (true) (Ex. If eof then put At end;)

rev a e sum s a emen s. x. a es o +sa es



14/117

Compiling a DATA Step

s mos anguages, e s ep s rs comp e en execu e .

Languages can be compiled, interpreted, SAS is a hybrid language

Some features from other languages

Some unique features

Sometimes difficult to separate compile versus execution events with SAS

DATA step compiler examines SAS statements for syntax data structures

generates an executable program

Uni ue to SAS com iler checkin for the existence of resources

Assumptions that it inserts into the source code.



15/117

Data Structures

e ng e a a n s ape nee s a a s ruc ures o o a a as processe .

All computer languages need to address this

Each language may name the structures differently

There is a lot of similarity in the way most languages store data.



16/117

A Typical SAS Job

BETH H 12 4822.12 982.10CHRIS H 2 233.11 94.12JOHN H 7 678.43 150.11

1234567890123456789012345678901234

input buffer

data softsale;infile rawin;

input name $1-10 division $12 years 15-16 sales 19-25 expense 27-34;run;

Name: NAME DIVISION YEARS SALES EXPENSE _ERROR_ _N_PROGRAM Type: CHAR CHAR NUM NUM NUM NUM NUMDATA Length: 10 1 8 8 8 8 8VECTOR Format:

Informat:Label:

DATASETDESCRIPTOR Name: NAME DIVISION YEARS SALES EXPENSEPORTION Type: CHAR CHAR NUM NUM NUM(DISK) Len th: 10 1 8 8 8

Value

Format:Informat:Label:

BETH H 12 4822.12 982.10DATASET CHRIS H 2 233.11 94.12DATA JOHN H 7 678.43 150.11PORTION



17/117

Raw File Buffers

s eps rea ng wr ng raw or non- a a nee memory u ers.

Needed to temporarily hold at least one input record at a time.

Also times when multiple lines of input can be held in buffers

Allows the program to logically read later rows before earlier ones.

Buffer contains the complete input and output record, regardless of

whether the INPUT statement reads all of the columns.

SAS datasets and RDMS (which usually appear as SAS datasets) do not

use raw buffers as the files are already in shape.



18/117

Raw File Buffers


1234567890123456789012345678901234

input buffer

data softsale;

infile rawin;input name $1-10 division $12 years 15-16 sales 19-25 expense 27-34;run;


Informat:Label:


Value





19/117

The LOGICAL PROGRAM DATA VECTOR (PDV)

secon memory area or a a man pu a on, convers on, re nemen .

Areas are needed for:

Inputting and input formatting (informatting) desired variables

Revising existing values

Computing new variables

System indicators and flags

Called Logical Program Data Vector (PDV).

Man lan ua es have a similar workin area. COBOL Workin Stora e .

Retained and Non-retained variables are stored in separate physical

program data vectors, together make Logical Program Data Vector

All variables referenced be automaticall defined in the PDV b com iler

using characteristics from the first reference of a variable.

Exam le: a emo=a e/12



20/117

The LOGICAL PROGRAM DATA VECTOR (PDV)


1234567890123456789012345678901234

input buffer

data softsale;

infile rawin;input name $1-10 division $12 years 15-16 sales 19-25 expense 27-34;run;


Informat:Label:

Value


Format:

Informat:Label:




21/117

The Final SAS Dataset

se - e n ng a ase .

The dataset descriptor contains attributes for all kept variables plus data

se a e ng n orma on.

The Dataset Data portion contains data values for all output rows andkept columns.

seu o-var a es are no ep .



22/117

The Final SAS Dataset


1234567890123456789012345678901234

input buffer

infile rawin;



Informat:Label:


Value

Format:

Informat:Label:




23/117

Variable Attributes

e comp er ass gns eac e ne var a e severa c arac er s cs.

Relative variable number

Position in the dataset

Name

Data type

Length in bytes

Informat

Format

Variable label

Flags to indicate dropping, retaining, handling of missing values for each

variable



24/117

Data Types and Conversion

va ues are one o wo a a ypes: numer c or c arac er.

Hundreds of different data types can be read or written via the PDV, only

wo s ore

In PDV, SAS Dataset every character value is stored as a native EBCDICor ASCII value with length between 1 and at least 32767

umer cs s ore as ou e prec s on oa ng po n va ues w eng

between 3 and 8 bytes.

Storing only two data types greatly simplifies things for SAS datasets moves e comp ca on o conver ng eren a a ypes pac e , nary,

etc.) to the INPUT and PUT statement along with appropriate

INFORMATS and FORMATS.

large numbers (or small) without overflow

Floating point does have minor mathematical issues of its own.



25/117

Data Types and Conversion


1234567890123456789012345678901234

input buffer




Informat:Label:


Value

Format:

Informat:Label:




26/117

Pseudo Variables

pec a var a es a e comp er crea es a are no a e o ou pu e.

Partial list:

_N_ contains the number of times the DATA step has looped.

_ERROR_ is set to 0 if there were no input errors, otherwise 1.

FIRST.variable showing beginning of control break

LAST.variable showing end of control break

_INFILE_ contains entire input buffer

Others can be requested by the programmer to:

Detect end of file

Access and alter system information



27/117

Pseudo Variables


1234567890123456789012345678901234

input buffer




Informat:Label:


Value

Format:

Informat:Label:




28/117

Output Datasets

sua y a a a e, u raw es an repor s can a so e crea e .

SAS data sets are self-defining via dataset descriptor

Much simpler operation for the programmer

SAS automatically builds the structures needed

Outputs the record at DATA step return.

In addition, all variables except dropped and pseudo variables are kept

Descriptor info can be printed with PROC CONTENTS, dictionary tables

Notes:

If a raw file or report is produced, buffers opposite those of INFILE are

used.

Raw files can pass info to other programs.



29/117

Questions About The Data Step

ra ona programm ng exper ence s app e s s ep, ques onsmight be:

Where do we write out records?What is looping?

en oes e program s op



30/117

Assumptions Made In The Data Step

e e au ac ons o e s ep eas y accommo a es mos programs.

All rows are read from files starting with the first record until last record.

Input values from a previous row are cleared before reading next row.

All variables referenced will be included on the output file.

All records will be included on the resulting output file.

All files should be opened at the beginning and closed at the end of step.

Programs should not continue to loop if no data is read in previous pass.



31/117


Immediately upon entry check for infinite looping

statements.

If any reading statement would read a record after end of file, step stops.

,

statement is executed, the current PDV contents (all columns for each

row) is output to the SAS data set being built.

pass.



32/117


no er v ew s a e comp er nser s e ue a c co e.

data softsale;

Check for looping initialize PDVheck for looping initialize PDV

infile rawin;if at EOF then stop

-Years 15-16 Sales 19-25Expense 28-34 State $36-37;

output to SAS Dataset

goto top of DATA step

run;

Notes:

These save effort and make DATA ste s virtuall infinite-loo roof.



33/117

Executing the DATA Step

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567

CHRIS H 2 233.11 94.12 WIMARK H 5 298.12 52.65 WISARAH S 6 301.21 65.17 MN

input Name $1-10 Division $12 Years 15-16

Sales 19-25 Expense 28-34 State $36-37;


run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer

Name Division Years Sales Expense StateLogical

Program

Data Vector

(Descriptor)Dataset work.softsale

Name Division Years Sales Expense State



34/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567



Sales 19-25 Expense 28-34 State $36-37;output to SAS Dataset

run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector





35/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .





36/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;

End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .




S


37/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .




E ti th DATA St


38/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567

CHRIS H 2 233.11 94.12 WI

MARK H 5 298.12 52.65 WISARAH S 6 301.21 65.17 MN



run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer CHRIS H 2 233.11 94.12 WI


Program

Data Vector . . .




E ti th DATA St


39/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program

Data Vector . . .CHRIS




E ti th DATA St


40/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program

Data Vector . . .CHRIS H






41/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program

Data Vector 2 . .CHRIS H






42/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer CHRIS H 2233.11

94.12 WI


Program

Data Vector 2 233.11 .CHRIS H






43/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program

Data Vector 2 233.11 94.12CHRIS H






44/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program

Data Vector 2 233.11 94.12CHRIS H WI






45/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program





. .



46/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program





. .



47/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program





. .



48/117


Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .




. .



49/117

g p

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;

End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .




. .



50/117

g p

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .




. .



51/117

g p

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer MARK H 5 298.12 52.65 WI


Program

Data Vector . . .




. .



52/117

g

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




Each row is handledthe same way.

Let's skip through

run; 1 2 3 4 8

1234567890123456789012345678901234567890...0


MARKs input step.


Program

Data Vector . . .




. .



53/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program

Data Vector 5 298.12 52.65 WIHMARK


Name Division Years Sales Expense StateCHRIS H 2 233.11 94.12 WI




54/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program



Name Division Years Sales Expense StateCHRIS

MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




55/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program




MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




56/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program




MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




57/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




58/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;

End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




59/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




60/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer SARAH S 6 301.21 65.17 MN


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




61/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End

1234567890123456789012345678901234567CHRIS H 2 233.11 94.12 WIMARK H 5 298.12 52.65 WISARAH S 6 301.21 65.17 MN



Each row is handled

the same way.

Let's skip through

run; 1 2 3 4 8

1234567890123456789012345678901234567890...0


SARAHS input step.


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




62/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program

Data Vector 6 301.21 65.17 MNSARAH S



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI




63/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program




MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI


SARAH S 6 301.21 65.17 MN


Fil f i


64/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0



Program




MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI


SARAH S 6 301.21 65.17 MN


Fileref ra in


65/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI


SARAH S 6 301.21 65.17 MN


Fileref rawin


66/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI


SARAH S 6 301.21 65.17 MN


Fileref rawin


67/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;

End





run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .



MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI


SARAH S 6 301.21 65.17 MN


Fileref rawin


68/117

Fileref rawin

data softsale;

init buffer, PDV

infile rawin;End


We're done!Go to the next step.




run; 1 2 3 4 8

1234567890123456789012345678901234567890...0

Input buffer


Program

Data Vector . . .

(Descriptor) Dataset work.softsaleName Division Years Sales Expense StateCHRIS

MARK

H

H

2

5

233.11

298.12

94.12

52.65

WI

WI


SARAH S 6 301.21 65.17 MN

Overriding Data Step Assumptions


69/117

o a programs scenar o a ove; can we a er e av or o e program

The FIRSTOBS= and OBS= system options begin and end logically after

.

The RETAIN compiler statement instructs the step to not initialize

variables.

.

The DELETE, subsetting IF statements, exit the DATA step early; never

reach OUTPUT.

.

DROP, KEEP statements, dataset options exclude or include columns.

GOTO and various DO groups alter the looping path for the program.


Retaining Data Values


70/117

ome mes var a es s ou no e c eare upon ex reen ry.

The RETAIN compiler statement:

Specifies listed variables should not be initialized

Can also set an initial value Iif first reference sets compiler attributes

Is essentially a flag set by compiler to tell execution dont clear

Can be coded anywhere in DATA Step

Variables read with SET MERGE or UPDATE MODIFY are retained.


Retaining Data Values


71/117

, .

MADISONBETH H 12 4822.12 982.10CHRIS H 2 233.11 94.12JOHN H 7 678.43 150.11

1234567890123456789012345678901234

data softsale;

infile rawin missover;

if _N_ = 1 then do;

input city $2-8;

delete; end;

input name $1-10 division $12 years 15-16 sales 19-25 expense 27-34;

retain city;

run;

MADISON

ame: _ _ _ _PROGRAM Type: CHAR CHAR NUM NUM NUM CHAR NUM NUMDATA Length: 10 1 8 8 8 7 8 8VECTOR Format:

Informat:Label:Flags: R D D

Value

DATASET

DESCRIPTOR Name: NAME DIVISION YEARS SALES EXPENSE CITYPORTION Type: CHAR CHAR NUM NUM NUM CHAR(DISK) Length: 10 1 8 8 8 7Format:

Informat:Label:


. .DATASET CHRIS H 2 233.11 94.12 MADISONDATA JOHN H 7 678.43 150.11 MADISONPORTION

Stopping Early


72/117

orma y o s run un , = or can s op s ep ear y.

Example: Stop after 50 records

data softsale;

infile rawin;

if N = > 50 then sto_ _

input name $1-10 division $12 years 15-16 sales 19-25

expense 27-34;

run;


Stopping Late


73/117

ome mes a program s ou con nue even e as recor was rea .Example: Calculate a percentage of total

DATA PCTDS;IF _N_ = 1 THEN

CONCAT DATASET

OBS NAME YEARS SALES

1 BETH 12 4822.122 CHRIS 2 233.113 JOHN 7 678.43

DO UNTIL(EOF);SET CONCAT(KEEP=NAME

SALES)END=EOF;

TOTSALES+SALES;

.5 ANDREW 24 1762.116 BENJAMIN 3 201.117 JANET 1 98.11

SET CONCAT(KEEP=NAME SALES);

SALESPCT=(SALES*100)/TOTSALES;RUN;

Name Sales TOTSALES SALESPCT

PDV


Outputting And Deleting Observations


74/117

e au s o ou pu e curren row s ep reac es mp e .

Statements to discard rows:

DELETE (usually after an IF) leaves the DATA step without OUTPUT.

False cases of Subsetting IF statements exit without OUTPUTting. RETURN exits the step but does OUTPUT.

Explicit OUTPUT statement OUTPUTs when executed, but no implied

OUTPUT at bottom.

You may prefer those positive statements such as Subsetting IF,OUTPUT etc.

You might use a negative statement such as DELETE to filter unwanted

rows.

Note:


WHERE also filters rows in engine, not in DATA step.

Outputting And Deleting Observations


75/117

xamp e: n y ou pu recor s w more an n a es.

.

data softsale;if no input last time thru then stop

initialize PDV

infile rawin;if at EOF then stop

-Division $12Years 15-16Sales 19-25Expense 28-34tate - ;

If sales > 4000;If sales not gt 4000 then goto top of DATA step


goto top of DATA stepp p

run;


WHERE also filters rows in engine, not in DATA step.

Losing the Crowd


76/117


Accumulating Totals in a DATA Step


77/117

e s ou e a e o wr e ormu as o coun or sum across o serva ons.

In a DATA step, read the following dataset and do the following:

Count all employees and print running counts.

Sum hours and print running sums.

fileref

rawin

JOE 40PETE 20STEVE .


Accumulating Totals in a DATA Step (continued)


78/117

infile rawin;

input Name $ Hours;

Ktr=ktr+1

JOE 40PETE 20Fileref

Hourstot=hourstot+hours;

run;

.TOM 35

raw n

proc print data=timecard;

title 'SOFTCO PAYROLL';

run;

SOFTCO PAYROLL

OBS Name Hours Ktr Hourstot

. .2 PETE 20 . .3 STEVE . . .

4 TOM 35 . .


Question: Why didn't the above program work correctly?

Accumulating Totals in a DATA Step (continued)

ng va ues across recor s as ree pro ems:


79/117

ng va ues across recor s as ree pro ems:

1. All variables are set to missing on every record.

2. Totals normally need to start at 0, not missing.

3. When a missing value is added to any value, the result is a missing value.

The SUM statement is a special assignment statement to add values

across observations.

SYNTAX:

variable+expression;

Notes:

SUM variables start at 0, not missing.

SUM variables are retained from pass to pass.


Any missing values encountered are ignored.

RETAIN and SUM function could also be used.

An Accumulation Solution


80/117

data timecard;infile rawin;input Name $ Hours;Ktr+1;

JOE 40PETE 20STEVE .TOM 35

filerefrawin

Hourstot+hours;

run;

proc print data=timecard;title 'SOFTCO PAYROLL'

run;

Name Hours Ktr HourstotflagsPDV




81/117



filerefrawin

Hourstot+hours;

run;


run;

Name Hours Ktr Hourstotflags RM RM

e acompiletime.

PDV0 0.




82/117



filerefrawin

Hourstot+hours;

run;


run;

Name Hours Ktr Hourstotflags RM RMPDV

. 0 0




83/117



filerefrawin

Hourstot+hours;

run;


run;


40 0 0JOE




84/117



filerefrawin

Hourstot+hours;

run;


run;


+ 1

PDV40 1 0JOE




85/117



filerefrawin

Hourstot+hours;

run;


run;


+ 40

PDV40 1 40JOE




86/117



filerefrawin

Hourstot+hours;

run;


run;


40 1 40JOE

1 JOE 40 1 40



d i d JOE 40


87/117



filerefrawin

Hourstot+hours;

run;


run;


PDV. 1 40

1 JOE 40 1 40



d t ti d JOE 40


88/117



filerefrawin

Hourstot+hours;

run;


run;


PDV . 1 40

1 JOE 40 1 40



data timecard; JOE 40


89/117


JOE 40PETE 20

STEVE .TOM 35

filerefrawin

Hourstot+hours;

run;


run;


PDV 20 1 40PETE

1 JOE 40 1 40



data timecard; JOE 40


90/117



filerefrawin

Hourstot+hours;

run;


run;


+ 1

PDV 20 2 40PETE

1 JOE 40 1 40



data timecard;i fil i JOE 40


91/117



filerefrawin

Hourstot+hours;

run;


run;


+ 20

PDV 20 2 60PETE

1 JOE 40 1 40



data timecard;i fil i JOE 40fil f


92/117



filerefrawin

Hourstot+hours;

run;


run;


PDV 20 2 60PETE

1 JOE 40 1 402 PETE 20 2 60



data timecard;infile rawin; JOE 4020fileref


93/117



filerefrawin

Hourstot+hours;

run;


run;


PDV . 2 60

1 JOE 40 1 402 PETE 20 2 60



data timecard;infile rawin; JOE 40PETE 20fileref


94/117



filerefrawin

Hourstot+hours;

run;


run;


PDV . 2 60

1 JOE 40 1 402 PETE 20 2 60





95/117


PETE 20STEVE .TOM 35

filerefrawin

Hourstot+hours;

run;


run;


PDV . 2 60STEVE

1 JOE 40 1 402 PETE 20 2 60





96/117

;infile rawin;input Name $ Hours;Ktr+1;


filerefrawin

Hourstot+hours;

run;


run;


+ 1

PDV . 3 60STEVE

1 JOE 40 1 402 PETE 20 2 60



data timecard;infile rawin; JOE 40PETE 20Fileref


97/117

;infile rawin;input Name $ Hours;Ktr+1;


FilerefRAWIN

Hourstot+hours;

run;


run;


+ .

PDV . 3 60STEVE

1 JOE 40 1 402 PETE 20 2 60





98/117

;input Name $ Hours;Ktr+1;


rawin

Hourstot+hours;

run;


run;


PDV . 3 60STEVE

1 JOE 40 1 402 PETE 20 2 603 STEVE . 3 60



data timecard;

infile rawin; JOE 40PETE 20fileref


99/117

;input Name $ Hours;Ktr+1;


rawin

Hourstot+hours;

run;


run;


PDV . 3 60

1 JOE 40 1 402 PETE 20 2 603 STEVE . 3 60





100/117

input Name $ Hours;Ktr+1;

STEVE .TOM 35

rawin

Hourstot+hours;

run;


run;


PDV . 3 60

1 JOE 40 1 402 PETE 20 2 603 STEVE . 3 60





101/117


STEVE .TOM 35

rawin

Hourstot+hours;

run;


run;


PDV 35 3 60TOM

1 JOE 40 1 402 PETE 20 2 603 STEVE . 3 60



data timecard;infile rawin;

$

JOE 40PETE 20fileref


102/117


STEVE .TOM 35

rawin

Hourstot+hours;

run;


run;


+ 1

PDV 35 4 60TOM

1 JOE 40 1 402 PETE 20 2 603 STEVE . 3 60



data timecard;infile rawin;i t N $ H

JOE 40PETE 20filerefi


103/117


STEVE .TOM 35

rawin

Hourstot+hours;

run;


run;


+ 35

PDV 35 4 95TOM

1 JOE 40 1 402 PETE 20 2 603 STEVE . 3 60



data timecard;infile rawin;input Name $ Hours;

JOE 40PETE 20STEVE

filerefrawin


104/117


STEVE .TOM 35

rawin

Hourstot+hours;

run;


run;


PDV 35 4 95TOM

1 JOE 40 1 402 PETE 20 2 603 STEVE . 3 60


4 TOM 35 4 95



JOE 40PETE 20STEVE

filerefrawin


105/117


STEVE .TOM 35

rawin

Hourstot+hours;

run;


run;




JOE 40PETE 20STEVE

filerefrawin


106/117


STEVE .TOM 35

rawin

Hourstot+hours;

run;


run;

SOFTCO PAYROLL

Obs Name Hours Ktr Hourstot

1 JOE 40 1 40

3 STEVE . 3 60

4 TOM 35 4 95


Another Accumulation Solution

e a n an e unc on can a so accumu a e correc y.

d t ti d


107/117

data timecard;

input Name $ Hours;

ktr=ktr+1;hourstot=sum hourstot,hours ;

retain Ktr Hourstot 0;

run;

proc print data=timecard;

title 'SOFTCO PAYROLL';run;


Compiler Instructions

.

In general, declarative statements Can be coded in any order


108/117

Can be coded in any order

.

Examples of these statements are:

LENGTH statement to set a variables internal length

INFORMAT to set input format

FORMAT to set out ut dis la format

LABEL to define a variable label

ATTRIB to define any or all of the above in one statement

DROP to indicate which variables are to be left behind on SAS file

KEEP to indicate which variables to include on the SAS file

RETAIN to set initial values and instruct SAS to never clear


Compiler Statements to Alter Variable Attributes

data softsale;

infile rawin;

length name $20; BETH H 12 4822 12 982 101234567890123456789012345678901234


109/117

length name $20;

attr division len th= 2


input buffer

format sales comma10.2;

input name $1-10 division $12 years 15-16 sales 19-25

expense 27-34;

run;

Name: NAME DIVISION YEARS SALES EXPENSE _ERROR_ _N_PROGRAM Type: CHAR CHAR NUM NUM NUM NUM NUMDATA Length: 20 2 8 8 8 8 8VECTOR Format: comma.

DATASET

Informat:Label:Flags: D D

Value

DESCRIPTOR Name: NAME DIVISION EXPENSEPORTION Type: CHAR CHAR NUM(DISK) Length: 20 2 8


BETH H 982.10DATASET CHRIS H 94.12


DATA JOHN H 150.11

PORTION


110/117

A Debugging Example

.

the problem?


111/117

1234567890123456789012345678901234

BETH H 12 4822.12 982.10CHRIS H 2 233.11 94.12

JOHN H 7 678.43 150.11

data softsale;infile rawin missover;input name $1-10 division $12 years 15-16 sales 19-25

-if sales > 4000;if division='h';

run;

NOTE: T e ata set WORK.SOFTSALE as 0 o servat ons an 5 var a es.


A Debugging Example

ow use o sp ay messages an va ues aroun e

statements.


112/117

data softsaleinfile rawin missover;input name $1-10 division $12 years 15-16 sales 19-25 expense 27-34;

putlog '$$$ before sales if ' _n_= sales= division=;if sales > 4000;putlog '$$$ before division if ' _n_= sales= division=;if division='h';run;

$$$ before sales if _N_=1 sales=4822.12 division=H$$$ before division if _N_=1 sales=4822.12 division=H

$$$ before sales if _N_=2 sales=233.11 division=H

= = = _ _


. .

Other Data Step Statements

, , ,

and process data in almost any form.


113/117

interfaces to the SAS macro system

much more much written about those features

beyond the scope of this paper to try to cover them all.

the DATA step is an extremely versatile and full featured programminglanguage.


Other Topics

,

languages.


114/117

auditable and more desirable.

Terrible DATA step code can be written.

Good design and adequate documentation make a well written program.

PROC SQL is also a great tool that adds that languages features to SAS.

SAS programs can be well written programs that can be used foreverything from one time programs to full fledged production programs.


Conclusions

features and extremely versatile features.


115/117


Not Good On the Flight Home


116/117


Contact UsQuestions, comments or to subscribe to the free Missing Semicolon

newslettter.


117/117

, ,

Steven J. First

President

(608) 278-9964

FAX (608) 278-0065

- .

www.sys-seminar.com

2997 Yarmouth Greenway Drive Madison, WI 53711


SAS Datastep and Program Data Vector Present

Documents

Transcript of SAS Datastep and Program Data Vector Present