Auditing access to data is quickly becoming a critical issue in the government and corporate worlds. DB2 auditing capabilities and the audit reporting process is covered in this cross-platform presentation. This case study examines the auditing capabilities of DB2, including audit policies, as well as audit reporting, management of the audited information, and some clever SQL statements to produce audit reports.

1

Dan Luksetich is a data scientist and senior DB2 DBA consultant. He works as a DBA, application architect, presenter, author, and teacher. Dan has been in the information technology business for over 29 years, and has worked with DB2 for over 24 years. He has been a COBOL and BAL programmer, DB2 system programmer, DB2 DBA, and DB2 application architect. His experience includes major implementations on z/OS, AIX, i Series, Windows, and Linux environments.

Dan's experience includes application design and architecture, business analytics, SQL consulting and education, database administration, complex SQL, SQL tuning, DB2 performance audits, replication, disaster recovery, stored procedures, UDFs, and triggers. Dan works everyday on some of the largest and most complex DB2 implementations in the world. He is a certified DB2 DBA for DB2 for z/OS and LUW, system administrator for z/OS, and application developer, and has worked on the teams that have developed the DB2 for z/OS certification exams. He is the author of several DB2 related articles as well as co-author of the DB2 9 for z/OS Certification Guide and the DB2 10 for z/OS Certification Guide.

Dan is an IBM Gold Consultant, IBM Champion for Data Management, and the chairman of the International DB2 Users Group Content Committee.

2

3

There is no doubt that the protection of data has become a major part of information management and information technology. Criminals and other nefarious elements are making headlines almost daily with malicious attempts to access sensitive data, steal identities, military and government information, and attack financial institutions. Individuals within organizations are also taking advantage of their authorized access to steal information. Protecting data, at all levels, is now part of (or should be part of) the daily business operations.

Enterprises are also being required to prove that they are protecting data and tracking access to data. Enter the auditing requirement.

4

Auditing needs to be performed at all levels. For the database administrator this means that access to DB2 has to be tracked and reported. This includes not only who accesses the data, but how they accessed it, when they accessed it, was it a read or a data change, and were they allowed to access the data. DB2 has built-in auditing capabilities. However, setting, maintaining, and reporting on the audit information provided by DB2 is much more that simply turning on a setting!

5

This presentation was inspired by an actual situation in which auditing was required for a government controlled database that contained some personal information about citizens of the government entity. This included personal identification information, as well as financial information. The protection of this information was mandated by law and violations of the law could result in a shutdown of the agency in charge, firing of employees, fines against employees, and potentially also conviction and imprisonment! This is serious!

All of the information was stored in DB2 databases located on z/OS and AIX operating systems. The DBA team had the responsibility of enabling the auditing on both platforms, as well as creating reports from the collected information. They also had to manage the output and make that output available to auditors. Setting up something like this is going to require a DBA with a lot of skills. Those skills will have to go beyond just DB2!

6

Management, auditors, and the application team determined which of the DB2 tables had to be audited as well as how they were to be audited on each of the platforms. On DB2 for z/OS we were to audit access to 4 tables, and the audit would include unauthorized access attempts, all reads, all changes to data, and the data changes themselves. On DB2 for LUW we were to audit access to 2 tables, and the audit would include unauthorized access attempts and all reads. No changes needed to be audited on the DB2 for LUW database due to the fact that it was a read-only database. We did , however, have to provide proof that the database was read only.

The audit reporting requirement was to produce the audit information on a once per day basis. There was an application team assigned the job of collecting and aggregating audit reports. However, that team was not prepared to handle the data so we decided to prepare the data in a variety of formats that they could dealwith. The reports produced had to be retained for six years, and protected themselves from unauthorized access.

7

Enhancements to the auditing capability of DB2 is continuing with each new release. This makes it easier for DBAs to establish auditing in a manner that is appropriate for their enterprise and their configuration. There are two main ways to enable auditing in DB2: via an audit trace or by establishing audit policies. The intention of the presentation is not to teach the audience about the audit settings in DB2, that’s what the manuals are for. The intention is to explain the possibilities and implications of various auditing techniques with DB2, as well as identify the fact that auditing goes well beyond simply enabling these features.

The auditing in the case study will draw on both the new and old school features of auditing in DB2.

8

DB2 for LUW provides an audit facility that is controlled via the db2audit system command. The system command is used to configure auditing at the instance level, start and stop auditing at the instance level, archive audit logs, and create reports from the audit log. If auditing is required at the instance level then the db2audit system command can be used for configuration. For individual database level auditing you will need to go beyond the system command and use SQL to create and activate audit policies.

Whether or not you are auditing at the instance level, database level, or both, the db2audit system command will be used to manage the generated audit information.

9

Audit tracing in DB2 for z/OS is similar in implementation to statistics and accounting traces in how they are implemented, controlled, and reported. Like all DB2 traces the trace records produced by the audit trace is sent to SMF, GTF, or a variety of other destinations. The default sends any audit trace records to SMF, from which it can be reported on by a variety of tools. There are 11 classes of audit traces that can be set in DB2 10 for z/OS to collect a wide variety of information on auditable events. These classes of trace are identified in the DB2 Command Reference manual.

In order for a table to be audited via the DB2 for z/OS audit trace auditing for that table must be activated. This is enabled by the AUDIT option of the CREATE TABLE or ALTER TABLE statement. The choices are NONE (the default), CHANGES (any change event to the table is recorded), or ALL (change events to and read events from the table are recorded). Audit trace records are written for any table without NONE as the option setting.

10

Audit policies are DB2 objects that provide for a more granular and flexible was of auditing objects and access in DB2 on both DB2 for z/OS and DB2 for LUW. The audit policy concepts and auditing categories that are a part of an audit policy are fairly similar in DB2 for z/OS and DB2 for LUW. This helps when communicating the auditing possibilities to management in a generalized fashion. The implementation of the audit policies are quite different between the platforms and so the DBAs involved need a diverse set of skills in which to set and manage audit policies.

11

There are 7 categories of audit policies and each audit policy created can be assigned one or more of these categories. You can create multiple audit policies, as well as attach the audit policies created to a variety of DB2 objects. In DB2 for z/OS the auditing via policies is slightly less flexible than in DB2 for LUW and so it benefits you greatly if you fully understand how they work on both platforms.

Creating audit policies in DB2 for LUW is on the database level and via SQL statements. Audit policies in DB2 for z/OS are at the subsystem or data sharing group level and create by inserting rows into the SYSIBM.SYSAUDITPOLICIES table, which is documented in the appendix of the DB2 for z/OS SQL Reference manual. The audit policy architecture for DB2 for z/OS is documented in the DB2 for z/OS Security Guide.

12

On DB2 for z/OS you activate an audit policy by issuing a START TRACE command that identifies the audit policy that is to be activated. There is no need to enable the AUDIT option for individual tables because the tables are all identified within the audit policy. One the trace is activated the policy is in effect and audit records are written to the trace destination.

The example on this slide shows the creation and activation of two audit policies used in DB2 for LUW in support of our case study. The requirement was to capture all authorized access to two tables and all unauthorized accesses. The was enabled by creating two audit policies, in one policy we utilized all categories but only the OBJMAINT and EXECUTE authorities applied. We were interested in EXECUTE for authorized access. That first policy was given a name and then the AUDIT statement was used to apply the policy and activate the audit for the two tables. The second policy was established with the CHECKING category and then applied at the database level since that is the only level to which it can be applied. This allowed us to capture all unauthorized access attempts.

13

In our case study we were required to audit access to 4 tables in DB2 for z/OS, but across all development, test, and production environments. We were also required to audit access to 2 DB2 for LUW tables in a single database. The audit under DB2 for z/OS included capturing all changes to data in the tables.

Given our situation with a separate service organization in control of the z Series servers it was more difficult to convince the them to use audit policies as audit policies were relatively new and unknown to the service organization staff. So, we went with classic old school audit traces in classes 1 and 5 and altered the target tables to AUDIT ALL and DATA CAPTURE CHANGES. The purpose of DATA CAPTURE CHANGES was to enable the use of a log analysis tool to report on the data changes to the target tables.

We were in complete control of the DB2 for LUW database and so as was described on the previous slide we established and activated some audit policies.

14

It is one thing to establish DB2 audits and begin collecting audit information. However, the real challenge comes with dealing with the output produced by these activated audits. There are two challenges in dealing with the audit output. The first is that any output produced has to be interpreted and formatted into a format that can be read by a human being or some sort of reporting tool. The second is that the amount of information that can be produced by audits can be quite significantly large and so you need to plan for ways to store this information as well as create processes that will manage the information.

15

DB2 audits can capture information about readers and writers to audited tables. DB2 audit can also capture the input parameters (host variables) used to query and/or update tables. The DB2 audit does not, however, capture the actual changes to the data values of columns in DB2 tables. So, if your auditing requirements state that these data values have to be captured then you’ll have to move beyond DB2 audit to do that. One solution is to use the DB2 transaction log to capture the changes to the tables. By ALTERing a table for DATA CAPTURE CHANGES you tell DB2 to capture full before and after images of rows that are being changed in a transaction. Since all the information is recorded in the log you could then use a tool to read the log to pull those values out. This is true for both DB2 for z/OS and DB2 for LUW.

16

Once you are gathering the audit information in the audit, transaction logs, and/or SMF records you now need to report on it. Enter the tools! There are a variety of commercially available and home grown tools that can be used to manage the audit output. In fact there are complete enterprise auditing packages available from various vendors. In order for our team to produce the required daily reports and files we had to use a combination of DB2 provided tools, purchased products, and some home grown coding and scheduling.

17

On DB2 for LUW we used all DB2 provided tools to extract the audit records, format them, and create a variety of reports. All of this was controlled via a single Korn shell script that was executed once per day around midnight. The script invoked some DB2 provided stored procedures to archive the audit log and then extract the audit log to produce delimited files that could be used to load a set of special audit reporting tables that are provided with DB2. The DB2 LOAD and EXPORT commands were then used to load these delimited files into the special audit tables and then produce various output files from the data in the tables. The data in the tables was replaced each day as the collection of our “human or tool” ready files and reports were the ultimate source of audit information for the end users. Special SQL was used within an EXPORT command to generate an XML file. A Standard EXPORT to a delimited file was good enough for our CSV requirement.

The db2audit command was used with the “extract” option to produce a standard DB2 audit report. This audit report was stored along with the daily XML and CSV files.

The process repeated every night and also generated empty files when there was nothing to report. A email was sent to the DBA team to report the success, failure, or generation of empty files to the DBA team.

18

19

Here is a snippet from the shell script that shows the use of the EXPORT command to produce a standard DB2 comma delimited file from the AUDIT_CHECKING audit table that was loaded earlier in the script.

20

With DB2 for z/OS our daily audit information is actually located in several places, including 1 SMF file and the transaction logs for 2 DB2 data sharing groups.

We used the Omegamon for DB2 batch reporter to produce a daily “human readable” audit report. In addition, in the same execution of Omegamon we used the FILE report command to generate load-ready files containing the audit information. These load-ready files were compatible with sample audit table definitions and LOAD utility control cards that were provided by Omegamon. These load-ready files were loaded into the target DB2 audit tables. After the LOAD we used the DB2 DSNTIAUL sample program to read the tables and format the daily XML and CSV files.

21

22

This is an example of the SQL that was used in an execution of DSNTIAUL to produce the XML formatted audit information. We wanted a day’s worth of audit information into a single document. This made for easier organization and management of the XML across platforms. The augmentation was provided by using DFSort to format the file and add the main document node as well as clean up the DSNTIAUL output a bit.

23

We had established a completely separate process to deal with the changed data information. This process use the IBM Log Analysis Tool to ready the various transaction logs for all our test and production DB2 data sharing groups. There are several other competing products available. The IBM LAT read the logs once per day and produced a detailed report of all changes to the audited tables. The tool also produced a file containing LOAD utility ready records as well as the LOAD utility control cards to load tables that would contain the change details. The tool was also capable of generating DDL to create the target tables for the change details.

Once the LAT extract ran each day we saved the detail report and loaded the generated change details into our target tables. SQL and DSNTIAUL was then used against these changed data tables to produce our daily XML and CSV files. It’s important to note that these changed data tables had to be protected just as much as the original audited tables.

24

25

26

This is an example of the SQL that was used in a DSNTIAUL execution to produce a CSV file. We had to use SQL over the UNLOAD to CSV utility due to the fact that all data needed to be in external format, the information from all tables combined into a single file, and in some instances to the joining of tables was required. This all goes beyond the capabilities of UNLOAD to a CSV.

27

There is yet another challenge when it comes to the information produced from the auditing of DB2 tables and that is the actual aggregation, management, and viewing of the audit output. This requires additional tools and/or personnel to manage this data. In many cases the data, in this case XML, CSV, and TXT files, had to be retained for several years. The tool purchased for this purpose in our case study did not work, and so we used a combination of various tools to view the reports and files.

All too often attention is placed on collecting and organizing the audit information and not enough attention is placed on the aggregation, management, and view of that information. Most of the pain involved in our case study revolved around accommodating management that did not know what they wanted or how they wanted it. This should normally be something that is resolved early in the project.

28

So, the lessons learned is that via DB2 and potentially various tools you can effectively capture audit information at various levels inside a DB2 database. This is quite clearly documented in the DB2 manuals and redbooks, along with the documentation for various vendors. However, the real challenge goes well beyond what is performed via DB2 audit. This includes establishing queries and reports that document who has access to the database and how that access is managed. Plus you have to remember that this is only the database component. What about the users, the applications, the application servers, the network. All of that needs to be audited.

29

It’s easy to set up and mange audits in DB2 on both the z/OS and LUW platform.The real challenge is in managing the quantity of data produced and getting it into a format that management can use for review. Specially planning has to be done well beyond the DB2 stuff. So, don’t forget to ask all the important questions:

• How often do you want the data?

• How long do we keep the data?

• What format do you want the data in?

• Have you reserved enough storage to hold all that data?

• Where is the data to be delivered?

The DB2 audit is useless unless there is a way to manage and read the output.

30

Dan Luksetich is a data scientist and senior DB2 DBA consultant. He works as a DBA, application architect, presenter, author, and teacher. Dan has been in the information technology business for over 29 years, and has worked with DB2 for over 24 years. He has been a COBOL and BAL programmer, DB2 system programmer, DB2 DBA, and DB2 application architect. His experience includes major implementations on z/OS, AIX, i Series, Windows, and Linux environments.

Dan's experience includes application design and architecture, business analytics, SQL consulting and education, database administration, complex SQL, SQL tuning, DB2 performance audits, replication, disaster recovery, stored procedures, UDFs, and triggers. Dan works everyday on some of the largest and most complex DB2 implementations in the world. He is a certified DB2 DBA for DB2 for z/OS and LUW, system administrator for z/OS, and application developer, and has worked on the teams that have developed the DB2 for z/OS certification exams. He is the author of several DB2 related articles as well as co-author of the DB2 9 for z/OS Certification Guide and the DB2 10 for z/OS Certification Guide.

Dan is an IBM Gold Consultant, IBM Champion for Data Management, and the chairman of the International DB2 Users Group Content Committee.

31