Splunk PCI 2.0 DataSource

Splunk App for PCI Compliance 2.0

Data Source Integration Manual

Generated: 5/11/2013 4:00 pm

Copyright © 2013 Splunk, Inc. All Rights Reserved

Table of ContentsIntroduction..........................................................................................................1

Overview...................................................................................................1

How to add a data source...................................................................................4 Create a technology add-on......................................................................4

Source types......................................................................................................20 Out-of-the-box source types....................................................................20

Examples............................................................................................................24 Generic example.....................................................................................24 Example 1: Blue Coat Proxy Logs..........................................................32 Example 2: OSSEC.................................................................................41

Resources...........................................................................................................53 FAQ.........................................................................................................53 Common Information Model Field Reference.........................................54 More resources.......................................................................................62

i

Introduction

Overview

The Payment Card Industry Data Security Standard or PCI DSS, is an industrystandard for all organizations that handle cardholder data. This data can includecredit cards, debit cards, ATM cards, and point of sale (POS) cards. The DataSecurity Standard is made up of twelve (12) requirements that businesses areexpected to comply with. The Splunk App for PCI Compliance gives the PCIcompliance manager visibility into PCI compliance-relevant data captured andindexed within Splunk.

The Splunk App for PCI Compliance's scorecards, reports, and correlationsearches are designed to present a unified view of PCI compliance acrossheterogeneous vendor data formats. Traditional approaches do so based onnormalizing the data into a common schema at time of data collection. Splunkprovides this unified view based on search-time mappings to a common set offield names and tags that can be defined at any time after the data is alreadycaptured, indexed, and available for ad hoc search.

These search-time mappings mean that you don't need to write parsers up frontbefore you can start collecting and searching the data. However, you do need todefine the field extractions and tags for each data format before the PCIcompliance scorecards, reports, and correlation searches will work on that data.These tags and field extractions for data formats are defined in technologyadd-ons. The Splunk App for PCI Compliance ships with an initial set of theseadd-ons. This manual explains how to create your own.

Technology add-ons contain the Splunk "knowledge" - field extractions, tags, andsource types - necessary to extract and normalize detailed information from thedata sources at search time and make the resulting information available forreporting. By creating your own technology add-ons, you can easily add new orcustom types of data and fully integrate them with the existing dashboards andreports within the Splunk App for PCI Compliance.

Once you have created a technology add-on, you can add it to your Splunk Appfor PCI Compliance deployment or post it to Splunkbase to share with others.

1

What is a technology add-on?

A technology add-on is a Splunk app that extracts knowledge from IT data sothat it can be processed by Splunk for PCI Compliance, as well as other appsthat leverage the Common Information Model (CIM). The technology add-on maypull data into Splunk or simply map data that is already coming in. Technologyadd-ons may conflict with or duplicate other Splunk apps that are already pullingin the same sort of data if they disagree on the source type. The differencebetween a technology add-on and another Splunk app is compliance with theCommon Information Model.

Note: The technology add-on will not require a user interface because reportingwill be handled by existing Centers and Searches in the Splunk App for PCICompliance.

Define a source type for the data

By default Splunk automatically sets a source type for a given data input. Eachtechnology add-on should have at least one source type defined for the data thatis captured and indexed within Splunk. This will require an override of theautomatic source type that Splunk will attempt to assign to the data source,because the primary source type must be set in the technology add-on in order toapply the right field extractions used by Splunk for PCI Compliance. A technologyadd-on can extrapolate key data within the raw text of logs to extract "fields," thatare fully compliant with the Common Information Model in the KnowledgeManager Manual.

Specifically, a technology add-on performs the following functions:

Capture and index the data: If necessary, the technology add-on canimport and source type the data into Splunk. This is not required if thedata is already in Splunk and source-typed properly.

•

Identify the relevant events that should be visible for security purposes(such as a successful login to a server).

•

Extract fields and aliases that match the CIM so that Notable Events canbe generated and dashboards will function properly.

•

Create tags to categorize the data (for example, tagging all dataindicating network communication with the tags "network" and"communicate").

•

Create any additional required fields that are not in the original datasource (such as fields that describe the vendor or product).

•

Normalize field values to a common standard (such as changing"Accepted public key" or "Success Audit" into "action=success").

•

2

Each technology add-on is designed for a specific data format, such as aparticular vendor's firewall or router. Once the technology add-on is created, datasources simply need to be assigned the corresponding source type for thetechnology add-on to begin processing the data.

Things you need to know to build a technology add-on

See the Common Information Model in the Knowledge Manager Manual, part ofthe core Splunk product documentation, for more information about these tasks:

How to create field extractions•

How to create new event types•

How to create tags•

How to create any additional fields you may need•

How to normalize field values•

How to map your data•

The Splunk App for PCI Compliance package includes a TA-template.zip in the$SPLUNK_HOME/etc/apps directory. This ZIP file includes templates for thecommon configuration or ".conf" files you will need to create your owntechnology add-on.

See the "Out-of-the-box Source Types" in this document for a list of tags andsource types that are already available with the Splunk App for PCI Compliance.

Available technology add-ons

Each Splunk App for PCI Compliance technology add-on is specific to a singletechnology, or portion of a technology that provides all the Splunk knowledgenecessary to incorporate that technology into the Splunk App for PCICompliance. You can use pre-packaged add-ons when available:

Technology add-ons for a number of common source types are bundledwith the Splunk App for PCI Compliance. Some of these add-ons mayneed to be configured for your environment. Each add-on contains aREADME file that details the required configurations.

•

3

How to add a data source

Create a technology add-on

Before you create a technology add-on, it is helpful to understand the format,structure, and meaning of the PCI compliance data that you wish to import intothe Splunk App for PCI Compliance.

Creating a technology add-on involves the following five steps:

Step 1: Capture and index the data

The first step in creating a technology add-on involves capturing the data andindexing the data within Splunk. Although this step doesn't require anythingbeyond the normal process of getting data into Splunk, decisions made duringthis phase can affect how a technology add-on identifies and normalizes the dataat search time.

The explicit tasks that are done in this step include:

Get the source data into Splunk•

Choose a name for your add-on directory•

Define a source type name to apply to the data•

Note: Changes made in Step 1 affect the indexing of data, so this step willrequire dumping the indexed data and starting over if the steps are doneincorrectly. Steps 2-4 do not affect indexing and can be altered while the app isrunning without restarting the server.

4

Get data into Splunk

Getting data into Splunk is necessary before the Splunk App for PCI Compliancecan normalize the data at search time and use the information to populatedashboards and reports. This step is highlighted due to the different ways thatsome data sources can be captured, thus resulting in different formats of data.As you develop a technology add-on, it is important to ensure that you areaccurately defining the method by which the data will be captured and indexedwithin Splunk.

Common data input techniques used to bring in data from PCIcompliance-relevant devices, systems, and software include:

Streaming data over TCP or UDP (for example, syslog on UDP 514)•

API-based scripted input (for example, Qualys)•

SQL-based scripted input (for example, Sophos, McAfee EPO)•

For more detailed information on getting data into Splunk, see the section "WhatSplunk can index" in the core Splunk product documentation.

The specific way in which a particular data set is captured and indexed in Splunkshould be clearly documented in the README file in the technology add-ondirectory.

Choose a folder name for the technology add-on

A technology add-on is packaged as a Splunk app and must include all of thebasic components of a Splunk app in addition to the components required toprocess the incoming data. All Splunk apps (including technology add-ons)reside in $SPLUNK_HOME/etc/apps.

The following table lists the files and folders of a basic technology add-on:

File/DirectoryName Description

default Contains the files related to the processing of the data

eventtypes.confDefines the event types (categories of events for the given sourcetype)

app.conf Describes the app and provides the ability to disable it

5

props.conf Contains attribute/value pairs that define how the data is processed

tags.conf Defines the tags

transforms.conf Contains additional attribute/value pairs required to process the data

local Includes configuration files that are custom to a particular installation

metadata Includes files that describe the app parameters

default.meta Defines the scope of the objects exported by the given app

lookups Contains the lookup CSV files

READMEDescribes the add-on, including configuration instructions, and thesupported product version

The transforms described in the transforms.conf file describe operations thatcan performed on the data. The props.conf file references the transforms so thatthey execute for a particular source type. In actual use, the distinction is not soclear because many of the operations in props.conf can be accessed directlyand avoid using transforms.conf.

See "Create and maintain search-time field extractions" in the core Splunkproduct documentation for more details.

Examples of these files are included in the TA-template.zip located in the$SPLUNK_HOME/etc/apps directory.

When building a new technology add-on, you must decide on a name for theadd-on folder. When choosing a name for your technology add-on folder, use thefollowing naming convention:

TA-<datasource>

The technology add-on folder should always begin with "TA-". This allows you todistinguish between technology add-ons and other add-ons within your Splunkdeployment. The <datasource> section of the name should represent the specifictechnology or data source that this technology add-on is for. Technology add-onsthat are shipped as part of the Splunk App for PCI Compliance follow thisconvention.

Examples include:

TA-bluecoat TA-cisco TA-snort

6

For additional details on deploying and configuring technology add-ons with theSplunk App for PCI Compliance, see the PCI Compliance Installation andConfiguration Manual.


By default Splunk automatically sets a source type for a given data input. Eachtechnology add-on should have at least one source type defined for the data thatis captured and indexed within Splunk. This will require an override of theautomatic source type that Splunk will attempt to assign to the data source. Thesource type definition is handled by Splunk at index time, along with linebreaking, timestamp extraction, and timezone extraction. (All other information isset at search time.) See the section "Specify source type for a source" in the coreSplunk product documentation.

You need to add a new source type for your technology add-on, making thesource type name match product so the technology add-on will work. Be awarethat this process overrides some default Splunk behavior. For explicit informationon how to define a source type within Splunk, see "Override automatic sourcetype assignment" in the core Splunk product documentation.

The source type name should be chosen to match the name of the product forwhich you are building a technology add-on (for example, "nessus"). Technologyadd-ons can cover more than one product by the same vendor (for example,"juniper"), and may require multiple source type definitions as a result. If the datasource for which you are building a technology add-on has more than one datafile with different formats (for example, "apache_error_log" and"apache_access_log") you may choose to create multiple source types.

These source types can be defined as part of the technology add-on ininputs.conf, just props.conf or using both props.conf and transforms.conf.These files must be created as part of the technology add-on. These files containonly the definitions for the source types that the technology add-on is designed towork with.

There are three ways to specify source types.

Let Splunk automatically define source types in the data1. Define source types in transforms.conf2. Define source types in inputs.conf3.

We recommend that you define your source types in inputs.conf. See"Configure rule-based source type recognition" in the core Splunk product

7

documentation for more information about source types.

Tip: In addition to the source type definition, you can add a statement that forcesthe source type when a file is uploaded with the extension set to the source typename. This allows you to import files for a given source type simply by setting thefile extension appropriately (for example, "mylogfile.nessus"). This statement canbe added to the props.conf file in theSplunk_TA-<technology_add-on_name>/default/props.conf directory as in thefollowing example:

[source:....nessus]sourcetype = nessus

Change the names "nessus" to match your source type.

Note: Usually, the source type of the data is statically defined for the data input.However, you may be able to define a statement that can recognize the sourcetype based on the content of the logs.

Confirm that your data has been captured

Once you have decided on a folder name for your technology add-on and definedthe source type(s) in the inputs.conf, just props.conf or using both props.confand transforms.conf, and props.conf files, your technology add-on can be givenlife and you can start collecting your data within Splunk. Turn on the data sourceeither by enabling the stream or the scripted input to begin data collection.Confirm that you are receiving the data and that the source type you've defined isworking appropriately by searching for the source type you defined in your data.

Note: Restart Splunk in order for it to recognize the technology add-on and thesource type defined in this step. After Splunk has been restarted, it automaticallyreloads the changes made to search time operations in props.conf andtransforms.conf.

8

If the results of the search are different than expected or no results are displayed,do the following:

Confirm that the data source has been configured to send data to Splunk.This can be validated by using Splunk to search for keywords and otherinformation within the data.

1.

If the data source is sent via scripted input, confirm that the scripted inputis working correctly. This may be more challenging to confirm, but can bedone by checking the data source, script, or other points of failure.

2.

If the data is indexed by Splunk, but the source type is not defined as youexpect, confirm your source type logic is defined appropriately and retest.

3.

Handling timestamp recognition

Splunk is designed to understand most common timestamps that are found in logdata. Occasionally, however, Splunk may not recognize a timestamp whenprocessing an event. If this situation arises, it is necessary to manually configurethe timestamp logic into the technology add-on. This can be done by adding theappropriate statements in the props.conf file. For specific information on how todeal with manual timestamp recognition, check "How timestamp assignmentworks" in the core Splunk product documentation.

Configure line-breaking

Multi-line events can be another challenge to deal with when creating atechnology add-on. Splunk handles these types of events by default, but onsome occasions Splunk will not recognize events as multi-line. If the data sourceis multi-line and Splunk has issues recognizing it, see the section "Configureevent linebreaking" in the core Splunk product documentation. Thedocumentation provides additional guidance on how to configure Splunk formulti-line data sources. Use this information in a technology add-on by makingthe necessary modifications to the props.conf file.

Step 2: Identify relevant PCI compliance events

After indexing, examine the data to identify the PCI compliance events that thesource includes and determine how these events fit into the PCI compliancedashboards. This step allows selection of the correct Common Information Modeltags and fields, so that data automatically populate the proper dashboards.

9

Understand your data and PCI compliance dashboards

To analyze the data, a data sample needs to be available. If the data is loadedinto Splunk, view the data based on the source type defined in Step 1. Determinethe dashboards into which this data might fit. See the "Reports" topic in theSplunk App for PCI Compliance Installation and Configuration Manual tounderstand what each of the reports and panels within the Splunk App for PCICompliance require and the types of events that are applicable. With thisinformation, review the data to determine what events need to be normalized foruse within the app.

Note: The reports in the Splunk App for PCI Compliance are designed to useinformation that typically appears in the related data sources. The data source forwhich the technology add-on is being built may not provide all the informationnecessary to populate the required dashboards. If this is the case, look furtherinto the technology that drives the data source to determine if other data isavailable or accessible to fill the gap.

Reviewing the data source and identifying relevant events should be relativelyeasy to do. This document assumes familiarity with the data and the ability todetermine what types of events the data contains.

After events and their contents have been reviewed, the next step is to match thedata to the application reports and dashboards. Review the dashboards directlyin the Splunk App for PCI Compliance or check the descriptions in the SplunkApp for PCI Compliance User Guide to see where the events might fit into thesereports(s) .

The dashboards in the Splunk App for PCI Compliance are grouped intoScorecards, Reports, and Audit:

Scorecards: Provide at-a-glance summary information about current PCIcompliance environment by requirement area. Scorecards presentreal-time views of the environment. At a glance, you can determine yourPCI compliance status in each of the requirement areas.

•

Reports: Provide a historical view of activity related to each of therequirement areas. With reports you can track your PCI compliance, byrequirement, over time.

•

Audit: The Audit dashboards provide a record of changes made in thePCI compliance environment to notable events, suppressions, forwarders,access to reports and scorecards, and consistency and security of data.

•

10

Within these areas, find the specific dashboard(s) that relate to your data. Insome cases, your data source may include events that belong in differentdashboards, or even in different areas.

Example:

Assume that you have decided to capture logs from one of your firewalls. Youwould expect that the data produced by the firewall will contain network trafficrelated events. However, in reviewing the data captured by Splunk you may findthat it also contains authentication events (login and logoff) and device changeevents (policy changes, addition/deletion of accounts, etc.).

Knowing that these events exist will help determine which of the dashboardswithin the Splunk App for PCI Compliance are likely to be applicable. In this case,the Network Traffic Activity Report, Firewall Rule Activity Report, andRequirement 1 Scorecard dashboards are designed to report on the eventsfound in this firewall data source.

Taking the time in advance to review the data and the application dashboardfunctionality will help make the next steps of defining the Splunk fields and tagseasier.

The product and vendor fields need to be defined. These fields are not includedin the data itself, so they will be populated using a lookup.

Static strings and event fields

Do not assign static strings to event fields because this prevents the fieldvalues from being searchable. Instead, fields that do not exist should be mappedwith a lookup.

For example, here is a log message:

Jan 12 15:02:08 HOST0170 sshd[25089]: [ID 800047 auth.info] Failedpublickey for privateuser from 10.11.36.5 port 50244 ssh2

To extract a field "action" from the log message and assign a value of "failure"to that field, either a static field (non-searchable) or a lookup (searchable) couldbe used.

For example to extract a static string from the log message, this informationwould be added to props.conf:

11

## props.conf[linux_secure]REPORT-action_for_sshd_failed_publickey =action_for_sshd_failed_publickey

## transforms.conf[action_for_sshd_failed_publickey]REGEX = Failed\s+publickey\s+forFORMAT = action::"failure"## note the static assignment of action=failure above

This approach is not recommended; searching for "action=failure" in Splunkwould not return theses events, because the text "failure" does not exist in theoriginal log message.

The recommended approach is to extract the actual text from the message andmap it using a lookup:

## props.conf[linux_secure]LOOKUP-action_for_sshd_failed_publickey = sshd_action_lookupvendor_action OUTPUTNEW actionREPORT-vendor_action_for_sshd_failed_publickey =vendor_action_for_sshd_failed_publickey

## transforms.conf[sshd_action_lookup]filename = sshd_actions.csv

[vendor_action_for_sshd_failed_publickey]REGEX = (Failed\s+publickey)FORMAT = vendor_action::$1

## sshd_actions.csvvendor_action,action"Failed publickey",failure

By mapping the event field to a lookup, Splunk is now able to search for the text"failure" in the log message and find it.

Step 3: Create field extractions and aliases

After you identify the events within your data source and determine whichdashboards the events correspond to within the Splunk App for PCI Compliance,now you create the field extractions needed to normalize the data and match thefields required by the Common Information Model and the PCI compliancedashboards and panels.

12

Use the following process to work through the creation of each required field, foreach class of event that you have identified:

Note: This process canbe done manually by editing configuration files or graphically by using theInteractive Field Extractor. See the "Examples" sections in this document formore detail on these options.

Choose event

In the Splunk Search app, start by selecting a single event or several almostidentical events to work with. Start the Interactive Field Extractor and use it toidentify fields for this event. Verify your conclusions by checking that similarevents contain the same fields.

Identify fields

Each event contains relevant information that you will need to map into theCommon Information Model. Start by identifying the fields of information that arepresent within the event. Then check the Common Information Model to seewhich fields are required by the dashboards where you want to use the event.

Note: See "Create reports" in the Splunk App for PCI Compliance Installationand Configuration Manual, which lists the fields required by each report (ordashboard). Additional information about the Common Information Model can befound in the "Common Information Model" topic in the Splunk documentation.

Where possible, determine how the information within the event maps to thefields required by the Common Information Model (CIM). Some events will havefields that are not used by the dashboard. It may be necessary to createextractions for these fields. On the other hand, certain fields may be missing orhave values other than those required by the CIM. Fields can be added ormodified, or marked as unknown, in order to fulfill the dashboard requirements.

Note: In some cases, it may turn out that the events simply do not have enoughinformation (or the right type of information) to be useful in the Splunk App for

13

PCI Compliance. The type of data being brought into the Splunk App for PCICompliance is not applicable to security (for example, weather information).

Create field extractions

After the relevant fields have been identified, create the Splunk field extractionsthat parse and/or normalize the information within the event. Splunk providesnumerous ways to populate search-time field information. The specific techniquewill depend on the data source and the information available within that source.For each field required by the CIM, create a field property that will do one ormore of the following:

Parse the event and create the relevant field (using field extractions)•

Example: In a custom application, Error at the start of a line meansauthentication error, so it can be extracted to an authentication field and tagged"action=failed".

Rename an existing field so that it matches (field aliasing)•

Example: The "key=value" extraction has produced "target=10.10.10.1" and"target "needs to aliased to "dest".

Convert an existing field to a field that matches the value expected by theCommon Information Model (normalizing the field value)

•

Example: The "key=value" extraction has produced "action=stopped", and"stopped" needs to be changed to "blocked".

Extract fields

Review each field required by the Common Information Model and find thecorresponding portion of the log message. Use the create field extractionstatement to extract the field. See "field extractions" for more information oncreating field extractions.

Note: Splunk may auto-extract certain fields at search time if they appear as"field"="value" in the data. Typically, the names of the auto-extracted fields willdiffer from the names required by the CIM. This can be fixed by creating fieldaliases for those fields.

14

Create field aliases

It may be necessary to rename an existing field to populate another field. Forexample, the event data may include the source IP address in the field src_ip,while the Common Information Model requires the source be placed in the "src"field. The solution is to create a field alias for "src" field that contains the valuefrom "src_ip". This is done by defining a field alias that will create a new fieldwith a name that corresponds with the name defined in the Common InformationModel.

See "field aliases" for more information about creating field aliases.

Normalize field values

You must make sure that the value populated by the field extraction matches thefield value requirements in the Common Information Model. If the value does notmatch (for example, the value required by the Common Information Model is"success" or "failure" but the log message uses "succeeded" and "failed") thencreate a lookup to translate the field so that it matches the value defined in theCommon Information Model.

See setting up lookup fields and tag and alias field values to learn more aboutnormalizing field values.

Verify field extractions

Once field extractions have been created for each of the security-relevant eventsto be processed, validate that the fields are in fact extracted properly. To confirmwhether the data is extracted properly, search for the source type.

Perform a search

Run a search for the source type defined in the technology add-on to verify thateach of the expected fields of information is defined and available on the fieldpicker. If a field is missing or displays the wrong information, go back throughthese steps to troubleshoot the technology add-on and figure out what is goingwrong.

Example: A technology add-on for Netscreen firewall logs is created by havingnetwork-communication events identified, a source type of "netscreen:firewall"created, and required field extractions defined.

To validate that the field extractions are correct, run the following search:

15

sourcetype="netscreen:firewall"

These events are network-traffic events. See the "Create reports" topic in theSplunk App for PCI Compliance Installation and Configuration Manual shows thatthis type of data requires the following fields: action, dvc, transport, src,dest, src_port, and dest_port.

Use the field picker to display these fields at the bottom of the events and thenscan the events to see that each of these fields is populated correctly.

If there is an incorrect value for a field, change the field extraction to correct thevalue. If there is an empty field, investigate to see whether the field is actuallyempty or should be populated.

Step 4: Create event types and tags

The Splunk App for PCI Compliance uses event types and tags to categorizeinformation and specify which dashboards an event belongs in. Once you havethe fields, you can create properties that tag the fields according to the CommonInformation Model.

Note: Step 4 is only required for Centers and Searches. Correlation searchescan be created with data that is not tagged.

Identify necessary tags

To create the tags, determine the dashboards where you want to use the dataand see which tags those dashboards require. If there are different classes ofevents in the data stream, tag each class separately.

To determine which tags to use, see the "Search view matrix" topic in the SplunkApp for PCI Compliance User Manual.

Create an event type

To tag the events, first create an event type that characterizes the class of eventsto tag, then tag that event type. An event type is defined by a search that is usedto describe the event in question. Event types have to be unique, each withdifferent names. The event types are created and stored in the eventtypes.conffile.

To create an event type for the technology add-on, edit thedefault/eventtypes.conf file in the add-on directory. Give the event type aname that corresponds to the name and vendor of the data source, such as

16

cisco:asa. Creating an event type actually creates a new field, which can betagged according to the Common Information Model.

Once the event type is created, you then create tags (for example"authentication", "network", "communicate", and so on) that are used to groupevents into categorizations. To create the necessary tags, edit the tags.conf filein the default directory and enable the necessary tags on the event type field.

Verify the tags

To verify that the data is being tagged correctly, display the event type tags andreview the events. To do this, search for the source type you created and use thefield discovery tool to display the field "tag::eventtype" at the bottom of eachevent. Then look at your events to verify that they are tagged correctly. If youcreated more than one event type, you also want to make sure that each eventtype is finding the events you intended.

Check PCI compliance dashboards

Once the field extractions and tags have been created and verified, the datashould begin to appear in the corresponding dashboard(s). Open eachdashboard you wanted to populate and verify that the dashboard informationdisplays properly. If it does not, check the fields and tags you created to identifyand correct the problem.

Note: Many of the searches within the Splunk App for PCI Compliance run on aperiodic scheduled basis. You may have to wait a few minutes for the scheduledsearch to run before data will be available within the respective dashboard.

Review the Search View Matrix in the Splunk App for PCI Compliance UserManual to see which searches need to run. Navigate to Manager > Searches tosee if those searches are scheduled to run soon.

17

Note: Searches cannot be run directly from the PCI compliance interface due toa known issue in the Splunk core permissions.

Step 5: Document and package the technology add-on

Once you have created your add-on and verified that it is working correctly,document and package the add-on for future reference, so that it is easy todeploy. Packaging your add-on correctly ensures that you can deploy and updatemultiple instances of the add-on.

Document the technology add-on

Edit or create the README file under the root directory of your technology add-onand add the information necessary to remember what you did and to help otherswho may use the technology add-on.

Note: If the Interactive Field Extractor has been used to build field extractionsand tags, they will be found under $SPLUNK_HOME/etc/users/$USER/.

The following is the suggested format for your README:

===Product Name Technology Add-on===

Author: The name of the person who created this technology add-on

Version/Date: The version of the add-on or the date this version wascreated

Supported product(s): The product(s) that the technology add-onsupports, including which version(s) you know it supports

Source type(s): The list of source types the technology add-onhandles

Input requirements: Any requirements for the configuration of thedevice that generates the IT data processed by the add-on

===Using this technology add-on===

Configuration: Manual/Automatic

Ports for automatic configuration: List of ports on which thetechnology add-on will automatically discover supported data (automatic configurationonly)

18

Scripted input setup: How to set up the scripted input(s) (ifapplicable)

Package the technology add-on

Next, prepare the technology add-on so that it can be deployed easily. Inparticular, you need to ensure that any modifications or upgrades will notoverwrite files that need to be modified or created locally.

First, make sure that the archive does not include any other files under theadd-on's local directory. The local directory is reserved for files that are specificto an individual installation or to the system where the technology add-on isinstalled.

Next, add a .default extension to any files that may need to be changed onindividual instances of Splunk running the technology add-on. This includesdynamically-generated files (such as lookup files generated by saved searches)as well as lookup files that users must configure on a per install basis. If youinclude a lookup file in the archive and do not add a .default extension, anupgrade will overwrite the corresponding file. Adding the .default extensionmakes it clear to the administrator that the file is a default version of the file, andshould be used only if the file does not exist already.

Finally, compress the technology add-on into a single file archive (such as a zipor tar.gz archive). To share the technology add-on, go to Splunkbase, clickupload an app and follow the instructions for the upload.

19

Source types

Out-of-the-box source types

This section provides a list of the data sources for which Splunk for PCICompliance provides out-of-the-box support. It also provides a list of the sourcetypes that are used for the different data sources and technology add-ons.

Source types are important because PCI Compliance uses source types as thebasis of understanding for all data coming in from a particular source. Sourcetypes need to be carefully defined so that they are not overloaded or misused.

When a supported data type is imported, the correct source type needs to beassigned to the data to ensure that data is recognized and parsed correctly byPCI Compliance. For example, events from a Juniper firewall must be assigned anetscreen:firewall source type for TA-juniper to recognize and parse themcorrectly.

To learn more about the supported data types and source types, see the "List ofpretrained source types" in the Splunk documentation. For more information onassigning source types to data inputs, see "About default fields" in the Splunkdocumentation.

The following table lists the data sources with out-of-the-box support in theSplunk App for PCI Compliance, along with the associated source type andtechnology add-on name:

Data Source Source type(s) Technology add-on

Wireless DevicesMotorola AirDefense wireless IDS airdefense TA-airdefense

ProxiesBlue Coat ProxySG bluecoat TA-bluecoat

Squid squid TA-squid

FirewallsJuniper NetScreen firewalls andIDP intrusion detection/preventionsystems

juniper:idp, netscreen:firewall,juniper:nsm:idp, juniper:nsm TA-juniper

fortinet TA-fortinet

20

Fortinet Unified ThreatManagement (UTM) systems

Palo Alto firewalls pan, pan:config, pan:system,pan:threat, pan:traffic TA-paloalto

Checkpoint firewalls checkpoint TA-checkpoint

IntrusionDetection/PreventionSystems

Juniper IDP juniper:idp, netscreen:firewall,juniper:nsm:idp, juniper:nsm TA-juniper

OSSEC host-based IntrusionDetection System (IDS) ossec TA-ossec

Snort network intrusionprevention and detection system(IDS/IPS)

snort TA-snort

McAfee firewall mcafee:ids TA-mcafee

WMI

WMI:LocalApplication,WMI:LocalSystem,WMI:LocalSecurity, WMI:CPUTime,WMI:FreeDiskSpace,WMI:LocalPhysicalDisk,WMI:Memory, WMI:LocalNetwork,WMI:LocalProcesses,WMI:ScheduledJobs, WMI:Service,WMI:InstalledUpdates, WMI:Uptime,WMI:UserAccounts,WMI:UserAccountsSID,WMI:Version

Splunk_TA_windows

Networking DevicesAlcatel network switches alcatel TA-alcatel

Common Event Format (CEF) cef TA-cef

flowd NetFlow collector flowd TA-flowd

FTP (File Transfer Protocol)servers vsftpd TA-ftp

Anti-virus / EndpointSoftwareMcAfee anti-virus mcafee:epo, mcafee:ids TA-mcafee

Symantec AntiVirus version 10and earlier.Use sep for version 11 andlater.

sav, winsav TA-sav

21

Symantec Endpoint Protection(SEP) host-based intrusiondetection/prevention systemand Symantec AntiVirusversion 11 and later.

sep, sep:scm_admin TA-sep

source::WinEventLog:Application WinEventLog:Application:trendmicro TA-trendmicro

Vulnerability ManagementSystemsnCircle IP360 vulnerabilitymanagement system ncircle:ip360 TA-ncircle

Nessus vulnerability scanner nessus TA-nessus

Nmap security scanner nmap TA-nmap

Operating SystemsSnare snare TA-windows

NTSyslog ntsyslog TA-windows

Monitorware monitorware TA-windows

Platform-specific Unixauthentication (security) logs.

dhcpd, linux_secure, aix_secure,osx_secure, syslog; TA-nix

Windows event, DHCP, andsystem update logs.

DhcpSrvLog, WindowsUpdateLog,WinRegistry, WinEventLog:Security,WinEventLog:Application,WinEventLog:System,fs_notification, scripts:InstalledApps,scripts:ListeningPorts

TA-windows

Includes the scripted inputsdeployed to deployment clients TA-deployment-apps

OtherIP2Location geolocation software TA-ip2location

Oracle database oracle TA-oracle

source::WinEventLog:Application WinEventLog:Application:rsa TA-rsa

Splunk access and authenticationlogs audittrail TA-splunk

Perfmon

PERFMON:CPUTime,PERFMON:FreeDiskSpace,PERFMON:Memory,PERFMON:LocalNetwork

TA-windows

22

Note about Cisco Add-ons

The TA-Cisco add-on has been removed from the Splunk App for PCICompliance package. The various Cisco apps and add-ons (Cisco IPS, CiscoFirewalls, Cisco CSA, Cisco ESA, Cisco WSA, Cisco MARS) provide thenecessary knowledge layers, without needing the general purpose one from theSplunk App for PCI Compliance. This makes it possible to download only thetechnology apps and add-ons needed in your environment.

These apps have been tested with Splunk for PCI Compliance:

Splunk for Cisco IPS http://splunk-base.splunk.com/apps/22292/splunk-for-cisco-ips

Splunk for Cisco Firewalls http://splunk-base.splunk.com/apps/22303/splunk-for-cisco-firewalls

Splunk for Cisco Client Security Agent http://splunk-base.splunk.com/apps/22304/splunk-for-cisco-client-security-agent

Splunk for Cisco IronPort Email Security Appliance http://splunk-base.splunk.com/apps/22305/splunk-for-cisco-ironport-email-security-appliance

Splunk for Cisco IronPort Web Security Appliance http://splunk-base.splunk.com/apps/22302/splunk-for-cisco-ironport-web-security-appliance

Splunk for Cisco MARS http://splunk-base.splunk.com/apps/22306/splunk-for-cisco-mars

These apps can be installed on the search head with Splunk for PCI Complianceand then partially disabled to prevent load.

To disable the Cisco searches, go to Manager > Searches and Reports,select the app name and disable all searches.

•

To disable their dashboards, go to Manager > User Interface > Views,select the app name and disable all views.

•

23

Examples

Generic example

A technology add-on maps data and extracts fields from a data feed for use withSplunk. The data needs to be expressed in security-relevant terms to be visiblewithin the Splunk App for PCI Compliance.

This example shows how to create a technology add-on to map a data feed foruse with Splunk for PCI Compliance. Mapping the data feed informs Splunk thatthe data in this source type will need these extractions to provide securitycontext. To create this knowledge, sample data for the new feed will be input andsource typed, field extractions will be created, tags will be created, and actionswill be created.

Before creating the technology add-on, you want to be familiar with Splunk forPCI Compliance and the data that you will be mapping (location, elements).Identify what portions of Splunk for PCI Compliance that the data will bepopulating (views, searches, dashboards).

For more information about the tags and fields needed for different views anddashboards, see the Search View Matrix in the Splunk App for PCI ComplianceUser Manual.

Process for creating the add-on is:

Get the data into Splunk• Set the source type• Create field extractions• Create eventtypes (if necessary)• Create tags (if necessary)• Prepare and populate the folder• Test and package the Add-on• Upload to Splunkbase•

Get the sample data into Splunk

With Splunk running and logged in as admin, upload a sample file of the data forwhich you want to create a technology add-on. For this example, we will workwith a hypothetical sample data source called "moof".

24

1. In Splunk go to to Manager > Data Inputs. Click Add New and upload asample file containing a representative number of records or connect to the datastreaming source. Be sure to use a manual sourcetype and select a name whichrepresents the data. The source type name is an important value that enablesSplunk to recognize that a data feed can be mapped with the knowledge in agiven technology add-on. Source typing tells Splunk what sort of extractions havebeen mapped for this data. The source type of data is set at index time, andcannot be changed after ingestion.

For introductory information about source typing, see "Why sourcetypes matter (alot)" in the core Splunk product documentation and Michael Wilde's article"Sourcetypes gone wild" on the Splunk blogs.

Splunk cannot index data without setting a source type, so automatic sourcetypes are set for data feeds during indexing. These automatic source typedecisions are permanent for a data input. For instance, if sample.moof isspecified as a data input file and the source type is set to moof, all future updatesfrom that file are set to the same source type. See "Override automatic sourcetype assignment" in the core Splunk product documentation to learn more aboutthis process. The list of source types that have already been mapped by Splunk("List of pre-trained source types") can be found in the core Splunk productdocumentation.

Adding a new data input through the user interface will establish alocal/inputs.conf file under the current application context (which is likely to belauncher or search). For instance, if the Welcome > Add data button were used,the inputs configuration file would be found at$SPLUNK_HOME/etc/apps/launcher/local/inputs.conf.

Important: For a production technology add-on, the sample data to be mappedshould be fed into Splunk in the same manner as live data, to reduce differencesin format and configuration. For example, if it will be arriving as a file, upload thesample data as a file; if it will be coming from TCP, UDP, or a script, then thesample data should be brought into Splunk by that method.

Note: Processing complex data feeds are beyond the scope of this chapter. Formore information about establishing a data feed to map with this process, see"What Splunk can monitor" in the core Splunk product documentation.

Create field extractions

Once the data is in Splunk and identified by a source type, view the data in theSearch app using sourcetype="moof" to test that the source type is working

25

correctly.

sourcetype="moof"

1. In the data sample, click the context menu which appears to the left of eachdata row and choose Extract Fields to launch the Interactive Field Extractor(IFX). The field Restrict field extraction to will contain the proper source typeby default, and should not be altered. Please review the "Interactive FieldExtractor manual" and "Manage search-time field extractions" before proceeding.

2. Refer to the fields needed for the views, dashboards, and searches in theSplunk App for PCI Compliance in Search View Matrix in the Splunk App for PCICompliance User Manual.

List the fields needed for this new technology add-on. For example, ifmoof is a type of firewall, its logs will be useful in Reports > NetworkTraffic Activity, which will lead to use of these fields:

•

action dvc transport src dest src_port dest_port

For each field required, use the IFX to construct a field extraction.

1. Select several samples from the data to paste into the "Example values for afield" field. If there are multiple instances and click Generate. A generatedpattern (regular expression or regex) string appears beneath the the examplevalue field, and extracted results appear to the left of the sample data. Extractedresults are highlighted with an "X" next to them. Click the "X" to delete unwantedmatches from the search and refine the regular expression.

Note: The data source may require parsing which is beyond the capabilities ofIFX to complete, in which case the data and the partially correct regularexpression can be copied into an external tool for further editing.

2. When the completed regular expression correctly matches the samples for thisfield, click Save and enter the name of the field.

Repeat this process for each of the fields needed to use the new data. Theseextractions are saved in$SPLUNK_HOME/etc/users/$USERNAME$/$APPNAME$/local/props.conf. For

26

instance, if this work is done as admin in the Search app, the path will be$SPLUNK_HOME/etc/users/admin/search/local/props.conf

Create event types (if necessary)

The Splunk App for PCI Compliance is not particularly sensitive to event type, butan event type is needed for tagging. Event types are used to differentiate thetypes of things that are done by the device which is logging data into Splunk. Forinstance, if moof is a firewall, it may have an event type of "moof_authentication"because it can authenticate a network connection.

Creating tags is only necessary for Centers and Searches in the Splunk for PCICompliance. Correlations searches can be created with data that is not tagged orevent typed, so this step is not necessary for all technology add-ons.

To create the event type:

1. In the Search app, look at the data using the source type you created. Forexample:

sourcetype="moof"

2. Select the proper event, and click the context menu and choose Createeventtype from the drop-down menu. In the Build Event Type editor, use thesource type as the event type. For instance, if moof logsauthentication_de_la_moof=true when it authenticates a connection, this eventshould be used.

3. Name the event type something descriptive (such as moof_authentication)and Save the event type.

The eventtype is stored in$SPLUNK_HOME/etc/users/$USERNAME$/$APPNAME$/local/eventtypes.conf. Forinstance, if this work is done as admin in the Search app, the path will be$SPLUNK_HOME/etc/users/admin/search/local/eventtypes.conf

Create tags (if necessary)

Tags are used to differentiate the results of events that are done by the devicewhich is logging into Splunk. For instance, if moof is a firewall, it may have a tagof "authentication" which normalizes "moof_authentication". Tags are used bySplunk to categorize information and specify the dashboards in which theinformation appears. For instance, a moof log of authentication_result=42 willnow produce an event type of moof_authentication which will produce a tag of

27

authentication. Using event types and tags allows unified searches acrossmultiple platforms with similar purposes, such as

tag::authentication="true"

To determine which tags are needed in the technology add-on, refer to theSearch View Matrix in the Splunk App for PCI Compliance User Manual. Make alist of the needed tags.

To create the tags:

1. Refer to Search View Matrix to make a list of which tags are needed. If we arelooking for firewall traffic, only two tags are needed network and communicate

2. View the event type using the Search app:

sourcetype="moof" eventtype="moof_authentication"

The new event type will be displayed and highlighted beneath the event; click thecontext menu to the right and select "tag eventtype=moof_authentication".

3. Enter the name of the tag and click Save.

Repeat this process for each of the tags needed to use the new data. These tagmodifications are also saved in$SPLUNK_HOME/etc/users/$USERNAME$/$APPNAME$/local/eventtypes.conf. Forinstance, if this work is done as admin in the Search app, the path will be$SPLUNK_HOME/etc/users/admin/search/local/eventtypes.conf

Prepare and populate the folder

Use the sample template files in $SPLUNK_HOME/etc/apps/TA-template.zip toprepare the technology add-on folder:

1. extract the template:

cd $SPLUNK_HOME/etc/apps/unzip TA-template.zip

2. Rename the extracted folder $SPLUNK_HOME/etc/apps/TA-template to a namethat reflects its new purpose, such as $SPLUNK_HOME/etc/apps/Splunk_TA-moof.

mv TA-template Splunk_TA-moof

3. Go to the Splunk_TA-moof directory.

28

cd Splunk_TA-moof

This will be the folder for the new technology add-on.

Edit inputs.conf if necessary

If this technology add-on will be responsible for feeding the data in, edit thedefault/inputs.conf file and specify the input mechanism, as well as setting asourcetype. For instance, this method would be used if moof logs were in binaryformat and needed to be translated with an external script before use in Splunk.If the data will be fed into Splunk directly (e.g. through file or data stream input),editing inputs.conf is not necessary. Review the$SPLUNK_HOME/etc/apps/$APPNAME$/local/inputs.conf file produced in [Get thesample data into Splunk] for example configuration.

Edit props.conf to set the sourcetype name

The source type is specified in the props.conf file. Multiple source type rules canbe set in this file to support different types of data feed. In this case, a simple fileextension rule is used to set the source type to moof.

To specify the feed and source type of our sample data, edit thedefault/props.conf file. List the source of the data in brackets, then set itssourcetype:

[source::/path/to/sample.moof]sourcetype = moof

Reference material for props.conf may be found in the "Splunk documentation".

Review the $SPLUNK_HOME/etc/apps/$APPNAME$/local/inputs.conf file producedin [Get the sample data into Splunk] for example configuration; setting sourcetype can be performed in inputs.conf or props.conf.

Edit transforms.conf and props.conf to prepare field extractions

The field extractions produced with IFX are saved in$SPLUNK_HOME/etc/users/$USERNAME$/$APPNAME$/local/props.conf. Forinstance, if this work was done as admin in the Search app, the path will be$SPLUNK_HOME/etc/users/admin/search/local/props.conf Each extraction issaved in the following format:

EXTRACT-$FIELD_NAME$ = $REGULAR_EXPRESSION$

29

This is strictly functional, but to provide a higher level of flexibility andmaintainability the technology add-on should split this form into atransforms.conf statement.

1. Copy each regular expression into default/transforms.conf in the followingformat:

[get_$FIELD_NAME$]REGEX = $REGULAR_EXPRESSION$FORMAT = $FIELD_NAME::$1

2. Reference each expression in default/props.conf in the following format:

REPORT-$FIELD_NAME$ = get_$FIELD_NAME$

Save both files. Now the technology add-on is prepared to do source typing ofthe data and extract the proper fields from it. This is sufficient for some basiccorrelation searches, but to fully utilize the data source event types and tagsshould be used as well.

Edit eventtypes.conf and tags.conf to prepare tags

The event types produced in the web console are saved in$SPLUNK_HOME/etc/users/$USERNAME$/$APPNAME$/local/eventtypes.conf. Forinstance, if this work is done as admin in the Search app, the path will be$SPLUNK_HOME/etc/users/admin/search/local/eventtypes.conf

Each event type is saved in the following format:

[authentication_moof]search = sourcetype="moof" "authentication_result="42"

1. Copy these event types into defaults/eventtypes.conf.

2. Create a new file defaults/tags.conf and enable tags for each event type:

[eventtype=authentication_moof]network = enabledcommunicate = enabled

Test and package the add-on

Restart Splunk, then open the Search app and verify that:

30

a. Source typing is working correctly - Go to the Summary screen, thesource type is listed under Sources.

b. Event types are showing up - Click on the source type and scroll downin the Field Discovery panel to "eventtype". The event type is listed.

c. Tags are displayed - Click the event type and scroll downtags::eventtype; the new tags are listed.

Next go into the Splunk App for PCI Compliance and check to see that thedashboards are populating correctly. Go to the dashboard you expect to bepopulated and check to see that the data is being displayed.

Review the technology add-on for complete coverage of the fields, event types,and tags required for the use cases and data sources it needs to support.

Document and package the technology add-on

Once you have created your add-on and verified that it is working correctly,document the add-on for future reference and package so it is easy to deployand share.

1. Edit the README file under the root directory of the technology add-on and addthe information necessary to remember what you did and to help others who mayuse the technology add-on. Note that the sample README fromTA-template.zip contains a suggested format.

2. Ensure that the archive does not include any files in the local directory. Thelocal directory is reserved for files that are specific to an individual installation orto the system where the technology add-on is installed.

3. Add a .default extension to any files that may need to be changed onindividual instances of Splunk running the technology add-on. This includesdynamically-generated files (such as lookup files generated by saved searches)as well as lookup files that users must configure on a per install basis. If youinclude a lookup file in the archive and do not add a .default extension, regularusage and/or upgrades will overwrite the corresponding file. Adding the .defaultextension makes it clear to the administrator that the file is a default version ofthe file, and should be used only if a more applicable file does not exist already.

31

Upload the new add-on to Splunkbase

Compress the technology add-on into a single file archive (such as a zip ortar.gz archive). To share the technology add-on, go to Splunkbase, login to anaccount, click upload an app and follow the instructions for the upload.

Example 1: Blue Coat Proxy Logs

This example shows how to create a technology add-on for Blue Coat web proxylogs from hosts running SGOS 5.3.3.8. (Earlier versions of Blue Coat requireadditional work to extract some of the required fields.)


Name this technology add-on Splunk_TA-bluecoat. Create the technologyadd-on folder at $SPLUNK_HOME/etc/apps/Splunk_TA-bluecoat. Extract thetemplate located at $SPLUNK_HOME/etc/apps/TA-template.zip to$SPLUNK_HOME/etc/apps/Splunk_TA-bluecoat.

Use the sample files in the template.zip to create the configuration files neededfor this technology add-on.

tags.conf• props.conf• transforms.conf• app.conf• eventtypes.conf•


To get started, set up a data input in order to get Blue Coat web proxy data intothe Splunk App for Enterprise Security. Blue Coat submits logs via syslog overport UDP\514, so use a network-based data input. Since this add-on will not beable to determine the source type based on the content of the logs (because it isusing network-based data input), the source type must be manually defined as"bluecoat".

Manually define the source type as "bluecoat".

32


This add-on only handles one type of logs, so we only need a single source type,which we will name "bluecoat". We define a source type in default/props.confof Splunk_TA-bluecoat:

[source::....bluecoat] sourcetype = bluecoat

[bluecoat]

Handle timestamp recognition

Looking at the events, we see that Splunk successfully parses the date and timeso there is no need to customize the timestamp recognition.

Configure line breaking

Each log message is separated by an end-line, therefore, we need to disable linemerging to prevent multiple messages from being combined. Line merging isdisabled by setting SHOULD_LINEMERGE to false in props.conf:

[source::....bluecoat] sourcetype = bluecoat

[bluecoat] SHOULD_LINEMERGE = false

Step 2: Identify relevant IT security events

Next identify which PCI compliance dashboards we want to have display the BlueCoat web proxy data.

Understand your data and PCI compliance dashboards

The data from the Blue Coat web proxy should go into the Proxy Center and theProxy Search dashboards.


Now create the field extractions that will populate the fields according to theCommon Information Model in the Knowledge Manager Manual. Before webegin, restart Splunk so that it will recognize the technology add-on and sourcetype defined in the previous steps.

33

Review the Common Information Model and the Search View Matrix in theSplunk App for PCI Compliance User Manual and determine that the Blue Coattechnology add-on needs to define the following fields to work with the ProxyCenter and Proxy Search:

Domain Sub-Domain Field Name Data TypeNetwork Protection Proxy action string

Network Protection Proxy status int

Network Protection Proxy src variable

Network Protection Proxy dest variable

Network Protection Proxy http_content_type string

Network Protection Proxy http_referer string

Network Protection Proxy http_user_agent string

Network Protection Proxy http_method string

Network Protection Proxy user string

Network Protection Proxy url string

Network Protection Proxy vendor string

Network Protection Proxy product string

Create extractions

Blue Coat data consists of fields separated by spaces. To parse this data we canuse automatic key/value pair extraction and define the name of the fields usingthe associated location.

Start by analyzing the data and identifying the available fields. We see that thedata contains a lot of duplicate or similar fields that describe activity betweendifferent devices. For clarity, create the following temporary naming convention tohelp characterize the fields:

Panel Description

s Relates to theproxy server

c

Relates to theclientrequestingaccess throughthe proxyserver

34

cs

Relates to theactivitybetween theclient and theproxy server

sc

Relates to theactivitybetween theproxy serverand the client

rs

Relates to theactivitybetween theremote serverand the proxyserver

Identify the existing fields and give them temporary names, listed here in theorder in which they occur.

1 date

2 time

3 c-ip

4 sc-bytes

5 time-taken

6 s-action

7 sc-status

8 rs-status

9 rs-bytes

10 cs-bytes

11 cs-auth-type

12 cs-username

13 sc-filter-result

14 cs-method

15 cs-host

16 cs-version

17 sr-bytes

18 cs-uri

19 cs(Referer)

20 rs(Content-Type)

35

21 cs(User-Agent)

22 cs(Cookie)

Once we know what the fields are and what they contain, we can map therelevant fields to the fields required in the Common Information Model (CIM).

Blue CoatField Field in CIM

date date

time time

c-ip src

sc-bytes bytes_in

time-taken duration

s-action action

sc-status status

rs-status N/A

rs-bytes N/A

cs-bytes bytes_out

cs-auth-type N/A

cs-username user

sc-filter-result N/A

cs-method http_method

cs-host dest

cs-version app_version

sr-bytes sr_bytes

cs-uri url

cs(Referer) http_referer

rs(Content-Type) http_content_type

cs(User-Agent) http_user_agent

cs(Cookie) http_cookie

Next create the field extractions. Since the Blue Coat data is space-delimited, westart by setting the delimiter to the single space character. Then define the orderof the field names. Add the following to defaults/transforms.conf in theSplunk_TA-bluecoat folder.

[auto_kv_for_bluecoat] DELIMS = " " FIELDS =

36

date,time,src,bytes_in,duration,action,status,rs_status,rs_bytes, bytes_out,cs_auth_type,user,sc_filter_result,http_method,dest, app_version,sr_bytes,url,http_referer,http_content_type, http_user_agent,http_cookie

Where:

DELIMS: defines the delimiter between fields (in this case a space)◊

FIELDS: defines the fields in the order in which they occur◊

Add a reference to the name of the transform to default/props.conf in theSplunk_TA-bluecoat folder to enable it:

[bluecoat] SHOULD_LINEMERGE = false REPORT-0auto_kv_for_bluecoat = auto_kv_for_bluecoat

Now define the product and vendor. Since these fields are not included in thedata, they will be populated using a lookup.

## props.confREPORT-vendor_for_bluecoat = vendor_static_bluecoatREPORT-product_for_bluecoat = product_static_proxy

## transforms.conf[vendor_static_bluecoat]REGEX = (.)FORMAT = vendor::"Bluecoat"

[product_static_proxy]REGEX = (.)FORMAT = product::"Proxy"

## bluecoat_vendor_info.csvsourcetype,vendor,productbluecoat,Bluecoat,Proxy

Note: Do not under any circumstances assign static strings to event fields as thiswill prevent these fields from being searched in Splunk. Instead, fields that do notexist should be mapped with a lookup. See "Static strings and event fields" in thisdocument for more information.

Now we enable the transforms in default/props.conf using REPORT lines tocall out the transforms.conf sections that will enable proper field extractions:

37

[bluecoat] SHOULD_LINEMERGE = false REPORT-0auto_kv_for_bluecoat = auto_kv_for_bluecoat REPORT-product_for_bluecoat = product_static_Proxy REPORT-vendor_for_bluecoat = vendor_static_Bluecoat

Generally, it is a good idea to add a field extraction that will process data that isuploaded as a file into Splunk directly. To do this, add a property todefault/props.conf that indicates that any files ending with the extensionbluecoat should be processed as Blue Coat data:

[source::.bluecoat] sourcetype = bluecoat

[bluecoat] SHOULD_LINEMERGE = false REPORT-0auto_kv_for_bluecoat = auto_kv_for_bluecoat REPORT-product_for_bluecoat = product_static_Proxy REPORT-vendor_for_bluecoat = vendor_static_Bluecoat


Now that the field extractions have been create, we need to verify that theextractions are working correctly. First, restart Splunk so that the changes can beapplied. Next, search for the source type in the Search dashboard:

sourcetype="bluecoat"

When the results are displayed, select Pick Fields and choose the fields thatought to be populated. Once the fields are selected, click the field name todisplay a list of the values appear in this field:

Step 4: Create tags

We created the field extractions, now we need to add the tags.

38


The Common Information Model requires that proxy data be tagged with "web"and "proxy":

Domain Sub-Domain Macro TagsNetwork Protection Proxy proxy web proxy

Create tag properties

Now that we know what tags to create, we can create the tags. First, we need tocreate an event type to which we can assign the tags. To do so, we create astanza in default/eventtypes.conf that assigns an event type of bluecoat todata with the source type of bluecoat:

[bluecoat] search = sourcetype=bluecoat

Next, we assign the tags in default/tags.confin our technology add-on:

[eventtype=bluecoat] web = enabled proxy = enabled

Verify the tags

Now that the tags have been created, we can verify that they are being applied.We search for the source type in the search view:

sourcetype="bluecoat"

Use the field picker to display the field "tag::eventtype" at the bottom of eachevent. Review the entries and look for the tag statements at the bottom of the logmessage.


Now that the tags and field extractions are complete, the data should show up inthe Splunk App for PCI Compliance. The extracted fields and the defined tags fitinto the Network Traffic Activity dashboard; therefore, the bluecoat data oughtto be visible on this dashboard. However, the bluecoat data will not beimmediately available since the Splunk App for PCI Compliance uses summaryindexing. It may take up to an hour after Splunk has been restarted for the datato appear. After an hour or so, the dashboard begins populating with Blue Coatdata:

39


Packaging the technology add-on makes it easy to deploy to other Splunk installsin your organization, or to post to Splunkbase for use in other organizations. Topackage a technology add-on you only need to provide some simpledocumentation an compress the files.

Document technology add-on

Now that the technology add-on is complete, create a README file under the roottechnology add-on directory:

===Blue Coat ProxySG technology add-on===

Author: John Doe Version/Date: 1.3 January 2012

Supported product(s): This technology add-on supports Blue Coat ProxySG 5.3.3.8

Source type(s): This technology add-on will process data that issource-typed as "bluecoat".

Input requirements: N/A


Configuration: Manual

To use this technology add-on, manually configure the data input withthe corresponding source type and it will optimize the dataautomatically.

40


Package up the Blue Coat technology add-on by converting it into a zip archivenamed Splunk_TA-bluecoat.zip. To share the technology add-on, go toSplunkbase, click upload an app and follow the instructions for the upload.

Example 2: OSSEC

This example shows how to create a technology add-on for OSSEC, anopen-source host-based intrusion detection system (IDS). This example issomewhat more complex than the Blue Coat example and shows how to performthe following additional tasks:

Use regular expressions to extract the necessary fields.•

Convert the values in the severity field to match the format required in theCommon Information Model

•

Create multiple event types to identify different types of events within asingle data source.

•


Get data into Splunk

To get started, set up a data input in order to get OSSEC data into the SplunkApp for PCI Compliance. OSSEC submits logs via syslog over port UDP\514,therefore, we would use a network-based data input. Once the add-on iscompleted and installed, it will detect OSSEC data and automatically assign it thecorrect source type if the data is sent over UDP port 514.


The name of this technology add-on will be Splunk_TA-ossec. Create thetechnology add-on folder at $SPLUNK_HOME/etc/apps/Splunk_TA-ossec byextracting $SPLUNK_HOME/etc/apps/TA-template.zip to$SPLUNK_HOME/etc/apps/Splunk_TA-ossec.

41


For this technology add-on use the source type ossec to identify data associatedwith the OSSEC intrusion detection system.

Confirm that the data has been captured

After the source type is defined for the technology add-on, set the source type forthe data input.

Handle timestamp recognition

Splunk successfully parses the date and time so there is no need to customizethe timestamp recognition.

Configure line breaking

Each log message is separated by an end-line; therefore, line-merging needs tobe disabled to prevent multiple messages from being combined. Line merging isdisabled by setting SHOULD_LINEMERGE to false in default/props.conf:

[source::....ossec] sourcetype=ossec

[ossec]

SHOULD_LINEMERGE = false

[source::udp:514] TRANSFORMS-force_sourcetype_for_ossec_syslog =force_sourcetype_for_ossec

Step 2: Identify relevant IT security events

Identify which events are relevant and need to be displayed in the Splunk App forPCI Compliance dashboards.

Understand the data and PCI Compliance dashboards

The OSSEC intrusion detection system proxy should provide data for theIntrusion Center dashboard.

42


The next step is to create the field extractions that will populate the fieldsaccording to the Common Information Model. Before we begin, restart Splunk sothat it will recognize the technology add-on and source type defined in theprevious steps.

After reviewing the Common Information Model and the DashboardRequirements Matrix, we determine that the OSSEC technology add-on needs toinclude the following fields:

Domain Sub-Domain Field Name Data TypeNetwork Protection Intrusion Detection signature string

Network Protection Intrusion Detection dvc int

Network Protection Intrusion Detection category variable

Network Protection Intrusion Detection severity variable

Network Protection Intrusion Detection src string

Network Protection Intrusion Detection dest string

Network Protection Intrusion Detection user string

Network Protection Intrusion Detection vendor string

Network Protection Intrusion Detection product string

Create extractions

OSSEC data is in a proprietary format that does not use key-value pairs or anykind of standard delimiter between the fields. Therefore, we will have to write aregular expression to parse out the individual fields. Below is an outline of a logmessage with the relevant fields highlighted:

We see that the severity field includes an integer, while the Common InformationModel requires a string. Therefore, we will extract this into a different field,severity_id, and perform the necessary conversion later to produce the severityfield.

43

Extracting the Location, Message, severity_id, signature and src_ip fields

Now, we edit default/transforms.conf and add a stanza that that extracts thefields we need above to:

[force_sourcetype_for_ossec] DEST_KEY = MetaData:Sourcetype REGEX = ossec\: FORMAT = sourcetype::ossec

[kv_for_ossec] REGEX = Alert Level\:\s+([^;]+)\;\s+Rule\:\s+([^\s]+)\s+- \s+([^\.]+)\.{0,1}\;\s+Location\:\s+([^;]+)\;\s*(srcip\:\s+(\d{1,3} \.\d{1,3}\.\d{1,3}\.\d{1,3})\;){0,1}\s*(user\:\s+([^;]+)\;){0,1}\s*(.*) FORMAT = severity_id::"$1" signature_id::"$2" signature::"$3" Location::"$4" src_ip::"$6" user::"$8" Message::"$9"

Then we enable the statement in default/props.conf in our technology add-onfolder:


[ossec] SHOULD_LINEMERGE = false REPORT-0kv_for_ossec = kv_for_ossec


Extract the dest field

Some of the fields need additional field extraction to fully match the CommonInformation Model. The Location field actually includes several separate fieldswithin a single field value. Create the following stanza in default/props.conf toextract the destination DNS name, destination IP address and original sourceaddress:


[ossec] SHOULD_LINEMERGE = false


44

[kv_for_ossec] REGEX = Alert Level\:\s+([^;]+)\;\s+Rule\:\s+([^\s]+)\s+- \s+([^\.]+)\.{0,1}\;\s+Location\:\s+([^;]+)\;\s*(srcip\:\s+(\d{1,3 }\.\d{1,3}\.\d{1,3}\.\d{1,3})\;){0,1}\s*(user\:\s+([^;]+)\;){0,1}\s*(.*) FORMAT = severity_id::"$1" signature_id::"$2" signature::"$3" Location::"$4" src_ip::"$6" user::"$8" Message::"$9"

[Location_kv_for_ossec] SOURCE_KEY = Location REGEX = ($([^$]+)\))*\s*(.*?)(->)(.*) FORMAT = dest_dns::"$2" dest_ip::"$3" orig_source::"$5"

Then we enable the statement in default/props.conf in the Splunk-TA-ossecfolder:


[ossec] SHOULD_LINEMERGE = false REPORT-0kv_for_ossec = kv_for_ossec, Location_kv_for_ossec


The "Location_kv_for_ossec" stanza creates two fields that represent thedestination (either by the DNS name or destination IP address). We need asingle field named "dest" that represents the destination. To handle this, addstanzas to default/transforms.conf that populate the destination field if thedest_ip or dest_dns is not empty:




[kv_for_ossec] REGEX = Alert Level\:\s+([^;]+)\;\s+Rule\:\s+([^\s]+)\s+- \s+([^\.]+)\.{0,1}\;\s+Location\:\s+([^;]+)\;\s*(srcip\:\s+(\d{1,3}\.\d{1,3 }\.\d{1,3}\.\d{1,3})\;){0,1}\s*(user\:\s+([^;]+)\;){0,1}\s*(.*) FORMAT = severity_id::"$1" signature_id::"$2" signature::"$3" Location::"$4" src_ip::"$6" user::"$8" Message::"$9"

45


[dest_ip_as_dest] SOURCE_KEY = dest_ip REGEX = (.+) FORMAT = dest::"$1"

[dest_dns_as_dest] SOURCE_KEY = dest_dns REGEX = (.+) FORMAT = dest::"$1"

Note: The regular expressions above are designed to match only if the string hasat least one character. This ensures that the destination is not an empty string.

Next, enable the field extractions created in default/transforms.conf by addingthem to default/props.conf. We want to set up our field extractions to ensurethat we get the DNS name instead of the IP address if both are available. We dothis by placing the "dest_dns_as_dest" transform first. This works becauseSplunk processes field extractions in order, stopping on the first one thatmatches.


[ossec] SHOULD_LINEMERGE = false REPORT-0kv_for_ossec = kv_for_ossec, Location_kv_for_ossec REPORT-dest_for_ossec = dest_dns_as_dest,dest_ip_as_dest


Extract the src field

We populated the source IP into the field "src_ip" but the CIM requires aseparate "src" field as well. We can create this by adding a field alias indefault/props.conf that populates the "src" field with the value in "src_ip":


[ossec]

46

SHOULD_LINEMERGE = false REPORT-0kv_for_ossec = kv_for_ossec, Location_kv_for_ossec REPORT-dest_for_ossec = dest_dns_as_dest,dest_ip_as_dest FIELDALIAS-src_for_ossec = src_ip as src


Normalize the severity field

The OSSEC data includes a field that contains an integer value for the severity;however, the Common Information Model requires a string value for the severity.Therefore, we need to convert the input value to a value that matches theCommon Information Model. We do this using a lookup table.

First, map the "severity_id" values to the corresponding severity string andcreate a CSV file in lookups/ossec_severities.csv:

severity_id,severity 0,informational 1,informational 2,informational 3,informational 4,error 5,error 6,low 7,low 8,low 9,medium 10,medium 11,medium 12,high 13,high 14,high 15,critical

Next, we add the lookup file definition to default/transforms.conf:




47

[kv_for_ossec] REGEX = Alert Level\:\s+([^;]+)\;\s+Rule\:\s+([^\s]+)\s+- \s+([^\.]+)\.{0,1}\;\s+Location\:\s+([^;]+)\;\s*(srcip\:\s+(\d{1,3 }\.\d{1,3}\.\d{1,3}\.\d{1,3})\;){0,1}\s*(user\:\s+([^;]+)\;){0,1}\s*(.*) FORMAT = severity_id::"$1" signature_id::"$2" signature::"$3" Location::"$4" src_ip::"$6" user::"$8" Message::"$9"




[ossec_severities_lookup] filename = ossec_severities.csv

Then we add the lookup to default/props.conf:


[ossec] SHOULD_LINEMERGE = false REPORT-0kv_for_ossec = kv_for_ossec, Location_kv_for_ossec REPORT-dest_for_ossec = dest_dns_as_dest,dest_ip_as_dest FIELDALIAS-src_for_ossec = src_ip as src LOOKUP-severity_for_ossec = ossec_severities_lookup severity_idOUTPUT severity


Define the vendor and product fields

The last fields to populate are the vendor and product fields. To populate these,add stanzas to default/transforms.conf to statically define them:


48



[kv_for_ossec] REGEX = Alert Level\:\s+([^;]+)\;\s+Rule\:\s+([^\s]+)\s+- \s+([^\.]+)\.{0,1}\;\s+Location\:\s+([^;]+)\;\s*(srcip\:\s+(\d{1,3}\.\d{1,3}\.\d{1,3 }\.\d{1,3})\;){0,1}\s*(user\:\s+([^;]+)\;){0,1}\s*(.*) FORMAT = severity_id::"$1" signature_id::"$2" signature::"$3" Location::"$4" src_ip::"$6" user::"$8" Message::"$9"




[ossec_severities_lookup] filename = ossec_severities.csv

[product_static_hids] REGEX = (.) FORMAT = product::"HIDS"

[vendor_static_open_source_security] REGEX = (.) FORMAT = vendor::"Open Source Security"

Next, enable the stanzas in default/props.conf:


[ossec] SHOULD_LINEMERGE = false REPORT-0kv_for_ossec = kv_for_ossec, Location_kv_for_ossec REPORT-dest_for_ossec = dest_dns_as_dest,dest_ip_as_dest FIELDALIAS-src_for_ossec = src_ip as src LOOKUP-severity_for_ossec = ossec_severities_lookup severity_id

49

OUTPUT severity REPORT-product_for_ossec = product_static_hids REPORT-vendor_for_ossec = vendor_static_open_source_security


Now that we have created the field extractions, we need to verify that they arecorrect. First, we need to restart Splunk so that it recognizes the lookups wecreated. Next, we search for the source type in the Search dashboard:

sourcetype="ossec"

When the results are displayed, we select Pick Fields to choose the fields thatought to be populated. Once the fields are selected, we can hover over the fieldname to display the values that have been observed:

Step 4: Create tags

We created the field extractions, now we need to add the tags.


The Common Information Model indicates that intrusion-detection data must betagged with "host" and "ids" to indicate that it is data from a host-based IDS.Additionally, attack-related events must be tagged as "attack". Below is theinformation from the Common Information Model:

Domain Sub-Domain Macro TagsNetwork Protection IDS ids_attack ids attack

Network Protection IDS host

50

Network Protection IDS ids

Create tag properties

Now that we know what tags we need, let's create them. First, we need to createthe event types that we can assign the tags to. To do so, we create an event typein eventtypes.conf that assigns an event type of "ossec" to all data with thesource type of ossec. We also create an additional event type, "ossec_attack",which applies only to those OSSEC events that are related to attacks:

[ossec] search = sourcetype=ossec #tags = host ids

[ossec_attack] search = sourcetype=ossec #tags = attack

Next, we assign the tags in tags.conf:

[eventtype=ossec] host = enabled ids = enabled

[eventtype=ossec_attack] attack = enabled

Verify the Data

Now that the tags have been created, verify that the tags are being applied.Search for the source type in the Search dashboard:

sourcetype="ossec"

Review the entries and look for the tag statements (they should be present underthe log message).


Now that the tags and field extractions are complete, the data should be ready toshow up in the Splunk App for PCI Compliance. The fields extracted and the tagsdefined should fit into the Intrusion Center dashboard; therefore, the OSSECdata ought to be visible on this view. The OSSEC data will not be immediatelyavailable in the dashboard since PCI Compliance uses summary indexing.Therefore, the data won't be available on the dashboard for up to an hour afterthe technology add-on is complete. After an hour or so, the dashboard shouldbegin populating with OSSEC data.

51


Once the technology add-on is completed and verified, create a readme file,clean up the directory structure, and tar or zip the add-on directory.

Document the technology add-on

Create a readme with the information necessary for others to use the technologyadd-on. The following file is created with the name README under the roottechnology add-on directory:

===OSSEC technology add-on===

Author: John Doe Version/Date: 1.2 January 2012

Supported product(s): This technology add-on supports Open Source Security (OSSEC) IDS 2.5

Source type(s): This technology add-on will process data that issource typed as "ossec".

Input requirements: Syslog data sent to port UDP\514


Configuration: Automatic Syslog data sent to port UDP\514 will automatically be detected asOSSEC data and processed accordingly.

To process data that is sent to another port, configure the data input with a source type of "ossec".


Package up the OSSEC technology add-on by converting it into a zip archivenamed Splunk_TA-ossec.zip. To share the technology add-on, go toSplunkbase, click upload an app and follow the instructions for the upload.

52

Resources

FAQ

I edited the transforms from Splunk Web and now I havecontent in the local directory. How do I merge this with thedefault content?

You can merge content from the local directory by copying the stanzas from thefile in local directory into the corresponding file in the default directory.

For example say you want to merge the following:

The local transforms file (local/transforms.conf) includes:

[bluecoat] SHOULD_LINEMERGE = false

[product_static_Proxy] REGEX = (.) FORMAT = product::"Proxy"

The default transforms file (default/transforms.conf) includes:

[bluecoat] REPORT-0auto_kv_for_bluecoat = auto_kv_for_bluecoat

The combined transforms file (in default/transforms.conf) would look like this:

[bluecoat] SHOULD_LINEMERGE = false REPORT-0auto_kv_for_bluecoat = auto_kv_for_bluecoat [product_static_Proxy] REGEX = (.) FORMAT = product::"Proxy"

Once you have migrated all the stanzas, make sure to delete the files in the localdirectory.

My source data is mostly tab-delimited, but the first threefields are space-delimited... these fields contain the date andtime, the log host, and the log type. What should I do?

53

Put these fields into one field called log_header and ignore it. The fields are notnecessary for the technology add-on to function.

Known Issues

Splunk fails to extract values spanning multiple lines

Splunk fails to automatically extract values when those values span multiplelines. The fields are extracted with the correct name but the value is left empty ifthe original value includes multiple lines.

To work around this issue, create a transform that extracts the entire field. Belowis a transform that extracts the multi-line field "message" for the source type"acme_firewall":

In transforms.conf:

[message_for_acme_firewall] REGEX = ,\s+message=\"(.*?)(\",\s+\S+\=) FORMAT = message::"$1"

Then, enable the transform in default/props.conf in the technology add-onfolder:

[acme_firewall]

REPORT-0 message_for_acme_firewall = message_for_acme_firewall

Common Information Model Field Reference

The tables below define the fields used in the Common Information Model. TheData Type column describes the type of data expected and the Descriptionprovides information about what the value of the field should represent and whichvalues are allowed (if the field is restricted to a defined set of potential values).

Access Protection

The Access Protection domain provides information about authenticationattempts and access control related events (login, logout, access allowed,access failure, use of default accounts, and so on).

54

Account Management

Field Name DataType Explanation

signature string Description of the change performed

src_nt_domain string The domain that contains the user that generated the accountmanagement event

dest_nt_domain string The domain that contains the user that is affected by theaccount management event

Authentication

FieldName

DataType Explanation

action string Must be either "success" or "failure".

app string The application involved in authentication. (for example, ssh, splunk,win:local).

dest string The target involved in authentication. (one of:dest_host,dest_ip,dest_ipv6,dest_nt_host)

src string The source involved in authentication. (one of:src_host,src_ip,src_ipv6,src_nt_host)

src_user string Privilege escalation events must include this field to represent the userwho initiated the privilege escalation.

user string The user involved in authentication. For privilege escalation events thisshould represent the user targeted by the escalation.

Endpoint Protection

The Endpoint Protection domain includes information about endpoints such asmalware infections, system configuration, system state (CPU usage, open ports,uptime, etc.), system update history (which updates have been applied), and timesynchronization information.

Authentication

FieldName


src string The client. Required for this entire Enterprise Security domain. (oneof: src_host, src_ip, src_nt_host)

55

Change Analysis

Field Name Data Type Explanationaction string Type of action performed on the resource

data string Data associated with the change event

dest stringThe host affected by the change(one of: dest_host, dest_ip, dest_ipv6,dest_nt_host)

msg string Message associated with the event

object string Name of affected object

object_attrs MV string Attributes changed on object, if applicable

object_category string Generic name for class of changed object

object_id stringUnique affected object ID as presented to system, if applicable(SID in Windows, UUID in UNIX if in use)

object_path string Full path to object , if applicable

severity string Severity of change, if applicable

status string Status of the change

user string User or entity performing the change (can be UID or PID)

user_type string Type of user performing change

Update

Field Name Data Type Explanationpackage string Name of the update that was installed

Malware


action string The outcome of the infection; must be one of "allowed","blocked", or "deferred".

product string The product name of the vendor technology (the "vendor" field)generating malware data. (for example, Antivirus, EPO)

signature string

The name of the malware infection detected on the client (the"src"), (for example, Trojan.Vundo, Spyware.Gaobot,W32.Nimbda).

Note: This field is a string. Please use the"signature_id" field for numbers.

dest string

56

The target affected or infected by the malware (for example,dest_host, dest_ip, dest_ipv6, dest_nt_host).

dest_nt_domain string The NT domain of the destination (the "dest_bestmatch").

src_nt_domain string The NT domain of the source (the "src")

vendor string The name of the vendor technology generating malware data.(for example, Symantec, McAfee)

file_path string The path of the file in the event (such as the infected ormalicious file)

file_hash string The cryptographic hash of the file associated with the event(such as the infected or malicious file).

user string The user involved in a malware event

file_name string The name of the file in the event (such as the infected ormalicious file)

product_version string The product version number of the vendor technology installedon the client (for example,. 10.4.3, 11.0.2)

signature_version string The current signature set (a.k.a. definitions) running on theclient. (for example, 11hsvx)

System Center


TotalMBytes int The amount of memory available on the system (the "src"field).

UsedMBytes int The amount of memory used on the system (the "src" field).

FreeMBytes int The amount of disk space available per drive or mount (the"mount" field) on the system (the "src" field).

mount string The drive or mount reporting available disk space (the"FreeMegabytes" field) on the system (the "src" field).

PercentProcessorTime int The percentage of processor utilization.

src_port int The TCP/UDP source port on the system

app string The running application or service (e.g., explorer.exe, sshd)on the system (the "src" field).

user string The User Account present on the system (the "src" field).

shell string The shell provided to the User Account (the "user" field)upon logging into the system (the "src" field).

setlocaldefs int The setlocaldefs setting from the SE Linux configuration

Startmode string The start mode of the given service (disabled, enabled, orauto).

sshd_protocol string The version of the sshd protocol.

57

selinux string Values from the selinux configuration file (disabled orenforcing)

selinuxtype string The SE Linux type (such as targeted)

updates int The number of updates the system (the "src" field) ismissing.

SystemUptime int The number of seconds since the system (the "src") hasbeen "up".

label string Human-readable version of the system uptime.

os stringThe name of the operating system installed on the host (the"src"). (for example, Microsoft Windows Server 2003,GNU/Linux)

kernel_release stringThe version of operating system installed on the host (the"src"). (for example, 6.0.1.4,2.6.27.30-170.2.82.fc10.x86_64)

Network Protection

Network Protection includes information about network traffic provided fromdevices such as firewalls, routers, and network based intrusion detectionsystems.

Change Analysis

Field Name Data Type Explanationdvc string The device that is directly affected by the change

action string The type of change observed.

user string The user that initiated the given change

command string The command that initiated the given change

Proxy


action string The action taken by the proxy.

status int The HTTP response code indicating the status of the proxyrequest (404, 302, 500, etc.)

src string The source of the network traffic (the client requesting theconnection)

dest string The destination of the network traffic (the remote host)

http_content_type string The content-type of the resource requested.

http_refer string The HTTP referrer used in requesting the HTTP resource.

58

http_user_agent string The user agent used when requesting the HTTP resource.

http_method string The HTTP method used in requested the resource (GET, POST,DELETE, and so on)

user string The user that requested the HTTP resource

url string The URL of the requested HTTP resource

vendor stringThe vendor technology of the generating Network Protectiondata; required for this entire Enterprise Security domain. (forexample, IDP, Proventia, ASA)

product stringThe product name of the vendor technology generating NetworkProtection data; required for this entire Enterprise Securitydomain. (for example, IDP, Proventia, ASA)

Traffic

FieldName


action string The action of the network traffic

transport string The transport protocol of the traffic observed (tcp, udp, icmp).

dvc string The name of the packet filtering device. (one of: dvc_host, dvc_ip,dvc_nt_host)

src string The source of the network traffic. (one of: src_host, src_ip, src_ipv6,src_nt_host)

dest string The destination of the network traffic. (one of: dest_host, dest_ip,dest_ipv6, dest_nt_host)

src_port int The source port of the network traffic

dest_port int The destination port of the network traffic

vendor stringThe vendor technology of the generating Network Protection data;required for this entire Enterprise Security domain. (for example, IDP,Proventia, ASA)

product stringThe product name of the vendor technology generating NetworkProtection data; required for this entire Enterprise Security domain. (forexample, IDP, Proventia, ASA)

Malware

FieldName


product stringThe product name of the vendor technology generatingNetworkProtection data; required for this entire Enterprise Securitydomain. (for example, IDP,Proventia,ASA)

severity string The severity of the NetworkProtection event. (i.e.,critical,high,medium,low,informational).

59

Note: This field is a string. Please use the "severity_id" fieldfor numbers.

vendor string The vendor technology generating NetworkProtection data; required forthis entire Enterprise Security domain. (e.g., Juniper,ISS,Cisco)

Intrusion Detection

FieldName


signature string

The name of the intrusion detected on the client (the "src")(for example,PlugAndPlay_BO, JavaScript_Obfuscation_Fre).

Note: This field is a string. Use the "signature_id" field fornumbers.

dvc string The device that detected the event

category string The category of the signature triggered

severity string

The severity of the Network Protection event. (for example, critical, high,medium, low, informational).

Note: This field is a string. Use the "severity_id" field fornumbers.

src string The source involved in attack detected by the IDS. (one of: src_host,src_ip, src_ipv6, src_nt_host)

dest string The destination of the attack detected by the IDS. (one of: dest_host,dest_ip, dest_ipv6, dest_nt_host)

user string The user involved with the attack detected by the IDS

vendor string

The vendor technology of the generating Network Protection data (forexample, IDP, Proventia, ASA.)

Required for this entire Enterprise Security domain.

product string

The product name of the vendor technology generating NetworkProtection data (for example, IDP, Proventia, ASA)

Required for this entire Enterprise Security domain.

ids_type string

The type of IDS (intrusion detection system) that generated the events.Must be one of "wireless", "network", "host",or "application"; use with theids and attack tags to indicate the event is related to an attack detectedby an IDS.

60

Packet Filtering

FieldName


action string The action the filtering device (the "dvc_bestmatch" field) performedon the communication. This must be either "allowed" or "blocked".

dest_port int The IP port of the packet's destination. (for example, 22)

dvc string The name of the packet filtering device. (one of: dvc_host, dvc_ip,dvc_nt_host)

rule string The rule which took action on the packet. (for example, 143)

src_port int The IP port of the packet's source. (for example, 34541)

Vulnerability

FieldName Data Type Explanation

signature string

The name of the vulnerability detected on the client(the "src" field).For example, SuSE Security Update: cupssecurity update.

os string

The operating system of the host containing thevulnerability detected on the client (the "src" field).For example, SuSE Security Update: cupssecurity update.

category string The category of the vulnerability discovered.

severity string The severity of the vulnerability discovered.

dest string

The host that has the vulnerability discoveredFor example one of:

dest_host,dest_ip,dest_ipv6,dest_nt_host).

cveCorresponds to an identifier provided in the CommonVulnerabilities and Exposures index,http://cve.mitre.org

For example: cve: CVE-1999-0002

Description: Buffer overflow in NFSmountd gives root access to remoteattackers, mostly in Linux systems.

bugtraqCorresponds to an identifier in the publicly availableBugtraq vulnerability database (searchable athttp://www.securityfocus.com/bid/)

For example: bugtraq: 52379

Description: Expat XML Parsing MultipleRemote Denial of Service Vulnerability

cert For example: cert: VU#636312

61

Corresponds to an identifier in the vulnerabilitydatabase provided by the US Computer EmergencyReadiness Team (US-CERT),http://www.kb.cert.org/vuls/

Description: Oracle Java JRE 1.7Expression.execute() andSunToolkit.getField() fail to restrictaccess to privileged code

msft Corresponds to a Microsoft Security Advisory number(http://technet.microsoft.com/en-us/security/advisory/)

For example: msft: 2743314

Description: Unencapsulated MS-CHAP v2Authentication Could Allow InformationDisclosure

mskb Corresponds to a Microsoft Knowledge Base articlenumber (http://support.microsoft.com/kb/)

For example: mskb: 2744850

Description: ImplementingPEAP-MS-CHAP v2 authentication forMicrosoft PPTP VPNs(http://support.microsoft.com/kb/2744850)

xref

A cross-reference identifier associated with thevulnerability. In most cases, the xref field will containboth a short name of the database beingcross-referenced in addition to the unique identifierused in the external database. In the followingexample "OSVDB" refers to the Open SourceVulnerability Database (http://osvdb.org).

For example: xref: OSVDB:299

Description: Microsoft Windows NetBIOSShares Access Control Weakness

More resources

Splunk App for PCI Compliance Overview:•

http://splunkbase.splunk.com/apps/PCI - Splunk App for PCI Compliance

Splunk App for PCI Compliance documentation•

http://docs.splunk.com/Documentation/PCI

Questions and answers (PCI Compliance-specific):•

http://splunk-base.splunk.com/tags/?q=pci

Questions and answers (General Splunk): http://answers.splunk.com•

General Splunk support: http://www.splunk.com/support•

Application link -http://splunk-base.splunk.com/apps/Splunk%20App%20for%20PCI%20Compliance

•

62

Splunk PCI 2.0 DataSource

Documents

Transcript of Splunk PCI 2.0 DataSource