Log Management Buyer ChecklistDr. Anton Chuvakin @ Security Warrior ConsultingOctober 2010

IntroductionThis checklist helps organizations looking to acquire log management tools clarify their needs and match their requirements to a log management tool’s capabilities. An Excel- based scorecard is also available to help evaluate and rate different solutions.

ChecklistThe correct tool should have the following core capabilities and functions:

1. Log collection

a. Support for collection and analysis of logs from the following log sources: network devices, popular server operating systems, security applications and appliances, databases, business applications, and any other log data deemed necessary for security, operations or compliance at your organization

b. Ability to acquire files, listen to syslog [UDP/TCP], get all Windows event logs from Windows Server 2003 to more recent versions, and fetch logs from databases

c. Support for collection of all of event types and all necessary details for each event logged and collected

d. Ability to grab log files produced by custom and industry-specific applications

e. Support for secure log collection when required and supported by a log source

f. Support for multiline logs such as log4j Java application logs and multibyte characters such as Chinese language characters

2. Retention

a. Securely and cost-effectively retain logs for at least one year (PCI DSS requirement)

b. Retention policies that are configurable per log source type, log source and group

3. First Impressions

a. An intuitive user interface for simple tasks

b. Ready-made reports and searches that can be used immediately

c. Clear guidance on how to use the tool for various tasks

4. Search

a. Fast! A simple keyword search over a day of logs should be nearly instant

b. Search across all collected data, from supported and custom applications

c. Support for partial-text (like Google) search as well as drill-down search with filters on event fields

d. Search archived data without extra steps (like copying archives to local storage or importing tables to a database)

e. Search by keyword, time stamp and log source name as well as other parameters

f. Allow searches to be easily saved as reports

5. Report

a. Fast! A simple canned report should take no more than a few minutes over days or weeks of log data

b. Provide out-of-the-box reporting for supported log sources without requiring users to parse log data into tables

c. Contain useful packaged reports that cover your log source and your needs, including regulatory compliance needs

d. Deliver specific packaged reports that cover authentication, privileged user activity, sensitive data access, system changes, account and trust modification, security issues and attacks, virus events, etc.

e. Provide a mechanism to easily create reports, without training and database skills or knowledge of the tool internals

f. Export, print or e-mail reports

6. Alert

a. Simple reporting on any keyword and log field

b. Support for filtering and forwarding log data to a real-time correlation engine

7. Compliance and Security

a. Provide an audit trail that allows organizations to prove that logs are being reviewed

b. Log all access to logs stored by the tool

c. Deliver secure archival as an option

d. Log deletion should be impossible; only the log retention mechanism can delete log data

ConclusionsUse this checklist and the Excel scorecard to compare and select log management tools for security, compliance and operations. To access the scorecard, go to: www.novell.com/docrep/2010/10/log_mgmt_evaluation_scorecard.xls