SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test...
Transcript of SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test...
![Page 1: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/1.jpg)
SIMPLE DATA AUTOMATION USING BASE SAS IN A WINDOWS ENVIRONMENT
Jennet Gambino
Bond Brand Loyalty
“You’re either the one that creates
the automation or you’re getting
automated.”
— Tom Preston-Werner
![Page 2: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/2.jpg)
Why do we automate data tasks?
• Increased productivity
• Reduced direct human
labor cost and expenses
• Improved quality and
consistency of output
![Page 3: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/3.jpg)
Client customer transactional files and survey
response data files originating from multiple
application systems will need to be
extracted, transformed and load (ETL) at
specific time daily.
CHALLENGES:
• Multiple files are coming from different systems at
different time (daily or hourly)
• Data processing business rules are massive
• Need email alerts to notify someone if any of files
are missing or if there are any issues with the content
![Page 4: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/4.jpg)
SAS Process
ReportsMFT Server
Reports/ Error Log
What a manual work flow looks like
Check that each file is
posted on the ftp
hourly
Other
Systems/
Applications
At specific time of the
day submit each SAS
code needed to
transform and generate
data files
Manual
Data Upload
![Page 5: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/5.jpg)
SAS Process
Reports / Emails
(Automated)MFT
Server
Reports / Error Log
Simple Workload Automation with SAS Processing
Scheduled Data Feed
(Automated)Other
Systems
Applications
Scheduled SAS Batch
Processing (Automated)
Email File
Issues
Scheduled
Data Feed
(Automated)
![Page 6: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/6.jpg)
SETUP REQUIREMENTS:
• Managed File Transfer (MFT) tasks
• SAS code (performing all the business rule/ data
processing)
• Windows Task Scheduler (server)
• Windows batch file script calling the necessary SAS
code and program (ex. Content of Step1_runSAS.bat)
"C:\PROGRAM FILES\SASHOME\SASFOUNDATION\9.X\SAS.EXE“ -SYSIN
T:\DATA_CLEANER.SAS -CONFIG
"C:\PROGRAM FILES\SASHOME\SASFOUNDATION\9.X\SASV9.CFG"
![Page 7: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/7.jpg)
THE WINDOWS TASK SCHEDULER
1. At the windows explorer, search “Task Scheduler”,
2. Choose Action in the menu, create basic task, fill all the necessary information,
name and description
Create a Basic Task
![Page 8: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/8.jpg)
THE WINDOWS TASK SCHEDULER
3. Next 2 screens, choose the frequency (and the time if applicable)
Create a Basic Task
![Page 9: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/9.jpg)
THE WINDOWS TASK SCHEDULER
3. Then, next 2 screens, will be the Action section settings. Enter the location of the
windows batch script.
Create a Basic Task
![Page 10: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/10.jpg)
THE WINDOWS TASK SCHEDULER
3. Final screen, review the summary of the tasks , then complete the settings by
clicking on the Finish button.
Create a Basic Task
![Page 11: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/11.jpg)
OUTPUT OF THE SCHEDULED TASK
• Formatted data files needed by other systems
• .LST file and output file containing all output information
• Emailed report(s) only if applicable
Create a Basic Task
![Page 12: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/12.jpg)
![Page 13: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/13.jpg)
AUTOMATION BEST PRACTICES:
• Always test all your SAS codes manually before
scheduling the automated tasks.
• Create the overall data flow chart, data processing and
schedule information as part of your documentation.
• Obtain all the necessary approvals from your clients.
Think about data security and privacy.
• Before going “LIVE” in the server, schedule a performance
test with your IT department.
![Page 14: SIMPLE DATA AUTOMATION USING BASE SAS IN A ... Group...AUTOMATION BEST PRACTICES: • Always test all your SAS codes manually before scheduling the automated tasks. • Create the](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f05610b7e708231d412ab3b/html5/thumbnails/14.jpg)
Me after a successful automation deployment.