File000138
-
Upload
desmond-devendran -
Category
Technology
-
view
362 -
download
3
Transcript of File000138
Module XXV– Log Capturing and Event Correlation
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
News: Intelligent Log Analysis May Beef up Security
Security logs could help detect and prevent security breaches, but analyzing their reports is so boring that they're underutilized.December 15, 2008
The massive job cuts caused by the recession will pose a huge threat to enterprise security because insider attacks, like disgruntled former employees, account for half of data breaches. Log monitoring and analysis tools provide poor protection from internal breaches because analyzing their reports is a tedious process, experts say.
LogRhythm may have solved this problem by adding the Intelligent IT Search feature to its log management tool. This automatically classifies and tags log entries for easy searching, conducts risk modeling and prioritizes sensitive issues, and puts a universal time stamp on all activities to make them easier to monitor.
Those features will make searches easier, which may help system administrators more rapidly detect breaches through searching the logs. According to the 2008 Verizon (NYSE: VZ) Business Data Breach Investigations Report, which covered a four-year time span, event monitoring or log analysis detected only four percent of breaches.
The technology is sound, and adoption rates have been high for some time, the Verizon report said. "In 82 percent of cases, the victim possessed the ability to discover the breach had they been more diligent in monitoring and analyzing event-related information available to them at the time of the incident. The breakdown is in the process."
And that process is tedious. Few IT administrators have the time to read logs frequently and look for unusual data activity, Prat Moghe, Tizor Systems' founder and chief technology officer, said in an article in Compliance Week. According to him, one retailer had an IT staffer spending six hours a day to look through logs.
Source: http://www.internetnews.com/
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Module Objective
• Computer Security Logs• Logs and Legal Issues• Log Management• Centralized Logging and Syslogs• Time Synchronization• Event Correlation• Log Capturing and Analysis Tools
This module will familiarize you with:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Module Flow
Computer Security Logs Log ManagementLogs and Legal Issues
Event CorrelationCentralized Logging
and SyslogsTime Synchronization
Log Capturing and Analysis Tools
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Computer Security Logs
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Computer Security Logs
Computer security logs contain information of the events occurring within an organization’s systems and networks
Security logs can be categorized as:
• Logs of Operating Systems (OSs) for servers, workstations, and networking devices (e.g., routers, switches)
Operating system logs:
• Logs of applications running on systems and servers such as email server, database server, etc.
Application logs:
• Logs of network and host-based security software
Security software logs:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Operating System Logs
OS logs are most beneficial for identifying or investigating suspicious activities involving a particular host
• Contains information of operational actions performed by OS components
Event Logs:
• Contains security event information such as successful and failed authentication attempts, file accesses, security policy changes, and account changes
Audit Logs:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Application Logs
Application Logs:
Client requests and server responses
Account information
Usage information
Significant operational actions
Application logs consist of all the events logged by the programs
Events that are written to the application log are determined by the developers of the software program
Windows Application Log
A Web Server Application Log
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Security Software Logs
Common types of network and host-based security software include:
• Antimalware Software • Intrusion Detection and Intrusion
Prevention Systems • Remote Access Software • Web Proxies • Vulnerability Management Software • Authentication Servers • Routers • Firewalls • Network Quarantine Servers
IDS Log
Antivirus Log
Firewall Log
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Router Log Files
Router stores log files in the router cache
Collect the bit stream image of the router cache for investigating log files
It provides detailed information of the network traffic on the Internet
It gives information on the attacks to and from the networks
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Honeypot Logs
The honeypot administrator is the only authorized user of honeypot
The logs that are found in honeypot are considered suspicious
These honeypot logs help forensic team to catch the attacker
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Linux Process Accounting
Linux Process Accounting tracks the commands that each user executes
The process tracking logfile is found at /var/adm, /var/log or /usr/adm
The tracked files can be viewed with lastcomm command
It enables process tracking by accton command or the startup (/usr/lib/acct/startup)
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Logon Event in Window
When the user logs on or off the computer, a logon event is generated
Logon on the security log is generated in the remote server when the user is connected to it
It can determine the attempts to log on interactively at servers
It examines the attacks launched from a particular computer
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Windows Log File
• sysevent.evt• secevent.evt• appevent.evt
Windows log files are stored in %systemroot%\system32\config\
Event viewer files can be checked in Control Panel Administrative Tools
• Kiwi Syslog for Windows• Event Reporter
Tools used for auditing these log files:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Configuring Windows Logging
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Analyzing Windows Logs
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Setting up Remote Logging in Windows
Deleting c:\winnt\system32\config\*.evt could erase the event-tracking logs
Windows does not support remote logging unlike Linux
NTSyslog enables remote logging in Windows
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Windows Log File: System Logs
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Windows Log File: Application Logs
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Logon Events That Appear in the Security Event Log
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Logon Events That Appear in the Security Event Log
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
IIS Logs
IIS logs all the server visits in log files located at:
• <%systemroot%>\logfiles
If proxies are not used, then IP can be logged
This command lists the log files:
• http://victim.com/scripts/ ..%c0%af../..%c0%af../..%c0%af../..%c0%af../..%c0%af../..%c0%af../..%c0%af../..%c0%af../winnt/system32/cmd.exe?/c+dir+C:\Winnt\system32\Logfiles\W3SVC1
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Maintaining Credible IIS Log Files
Most of the network administrators have encountered serious Web server intrusions that have resulted in legal action
Often, IIS logs are considered as the primary evidence used to track down Web intruders
IIS logs can provide convincing evidence of your argument if their credibility is challenged in court
Protect and maintain the accuracy, authenticity, and accessibility of logs to make them reliable and admissible evidence
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Log File Accuracy
Log file accuracy is proving that log file data truly represents the activity on the Web server
Even the smallest inaccuracy can bring into question the validity of the entire set of data
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Log Everything
For logging everything , configure your IIS logs to record every available field
While few administrators see value in storing this extra information, every field has some significance in forensic investigation
Gathering information about Web visitors helps establish that an attack came from a specific computer system or logged in user
For example, suppose a defendant claims a hacker had broken into his computer and installed a backdoor proxy server, then used that backdoor proxy to attack other systems; in this case logging every server activity may help investigators in finding the origin of traffic and perpetrator of the crime
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Keeping Time
• Key: HKLM\SYSTEM\CurrentControlSet|Services\W32Time\Parameters\Setting: TypeType: REG_SZValue: NTP
• Key: HKLM\SYSTEM\CurrentControlSet|Services\W32Time\Parameters\Setting: NtpServerType: REG_SZValue: ntp.xsecurity.com
On a standalone server, you can synchronize to an external source by setting the following registry entries:
Synchronize your IIS servers to an external time source using the Windows Time Service
If you use a domain, the Time Service will automatically be synchronized to the domain controller
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
UTC Time
IIS records logs using UTC time
It helps in synchronization issues , when running servers in multiple time zones
Windows calculates UTC time by offsetting the value of the system clock with the system time zone
The only way to be sure the UTC time is correct is to ensure that the local time zone setting is accurate
If your server is set at UTC -0600, then the first log entries should appear around 18:00 (00:00 - 06:00 = 18:00)
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
View the DHCP Logs
The DHCP logs are saved in the C:\WINNT\System32\DHCP folder on DHCP servers
Actual location depends on where Microsoft Windows NT or Microsoft Windows 2000 is installed
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
DHCP Logs
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
ODBC Logging
ODBC logging is a record of a fixed set of data properties in a database that complies with ODBC, such as Microsoft Access or Microsoft SQL Server
It includes the IP address of the user, user name, request date and time, HTTP status code, bytes received, bytes sent, action carried out, and the target file
It specifies the database to be logged to, and sets up the database to receive the data
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Logs and Legal Issues
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Legality of Using Logs
First, the logs must be created reasonably and contemporaneously with the event
Log files should not be tampered with
Someone with knowledge of the event must record the information
In this case, the recording is being done by a program; the record therefore reflects the prior knowledge of the programmer and system administrator
Logs must be kept as a regular business practice
Random compilations of data are not admissible
Logging systems instituted after an incident do not qualify under the business records exception
Keep regular logs to use them as evidence later
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Legality of Using Logs (cont’d)
A “custodian or other qualified witness” must testify to the accuracy and integrity of the logs
The custodian need not be the programmer who wrote the logging software; however, he or she must be able to offer testimony on what sort of system is used, where the relevant software came from, how and when the records are produced, etc.
It is necessary to offer testimony for the reliability and integrity of the hardware and software platform used, including the logging software
A record of failures or security breaches on the machine creating the logs will tend to impeach the evidence
Log entries of the machine claimed to be penetrated are considered suspicious
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Legality of Using Logs (cont’d)
In a civil lawsuit against the attackers, anything in your own records that would tend to exculpate the defendants can be used against you
Your own logging and monitoring software must be made available to them, to permit them to attack the credibility of the records
But under certain circumstances, if you can show that the relevant programs are trade secrets, you may be allowed to keep them secret, or disclose them to the defense, only under a confidentiality order
The original copies of any files are preferred
A printout of a disk or tape record is considered to be an original copy, unless and until judges and jurors come equipped with USB/SCSI interfaces
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Records of Regularly Conducted Activity as Evidence
“A memorandum, report, record, or data compilation, in any form, of acts, events, conditions, opinions or diagnoses, made at or near the time by, or from information transmitted by, a person with knowledge, if kept in the course of a regularly conducted business activity, and if it was the regular practice of that business activity to make the memorandum, report, record, or data compilation, all as shown by the testimony of the custodian or other qualified witness, unless the source of information or the method or circumstances of preparation indicate lack of trustworthiness”
Rule 803, Federal Rules of Evidence
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Laws and Regulations
• Federal Information Security Management Act of 2002 (FISMA)• Gramm-Leach-Bliley Act (GLBA)• Health Insurance Portability and Accountability Act of 1996 (HIPAA)• Sarbanes-Oxley Act (SOX) of 2002• Payment Card Industry Data Security Standard (PCI DSS)
The following regulations, standards, and guidelines define organizations’ needs for log management:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Log Management
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Log Management
• Log Generation• Log Analysis and Storage• Log Monitoring
Log management infrastructure typically comprises the following three tiers:
Log management includes all the processes and techniques used to collect, aggregate, and analyze the computer-generated log messages
Log management systems consist of the hardware, software, network and media used to generate, transmit, store, analyze, and dispose of log data
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Functions of Log Management
• Log parsing • Event filtering • Event aggregation • Log rotation • Log archival and retention • Log compression • Log reduction • Log conversion • Log normalization • Log file integrity checking • Event correlation • Log viewing • Log reporting • Log clearing
Log management system performs the following functions:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Challenges in Log Management
Detecting variety of intrusions attempted on your network
Overall Internet bandwidth usage of the enterprise network
Identifying who/when/what activities inside your network
Individual employees’ non-business web usage
Audit and regulatory compliance requirements
Monitoring enterprise policy implementation of access to internal network resources
Threats and user activities at server and SQL applications
Regulatory compliance and audit requirements
Forensic analysis
Troubleshooting
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Centralized Logging and Syslogs
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Central Logging Design
ConversationalMonitor System Portal
StreamingMedia
JavaApplication
SyslogSyslog
Backup Log Server Mail Apache
Swatch
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Centralized Logging Setup
Router
IDS
Host
Firewall
AGENTS
Oracle Database
Reporting Tool: Real-Time Analysis: Forensics Report
NF Engine: Event Aggregation and
Correlation
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Steps to Implement Central Logging
1.• Secure the location of log server
2.• Turn off all services that are running for security purpose
3.• Turn off all Internet Daemon services such as Syslog and Secure Shell
4.• Disable Remote Procedure Call (RPC) services
5.• Disable all unnecessary accounts
6.• Specify the time on all devices
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Syslog
Syslog is a client/server protocol standard for forwarding log messages across an IP network
The term syslog refers to both the syslog protocol and the application or library sending syslog messages
Syslog sender sends log message to the syslog receiver also known as syslogd, syslog daemon or syslogserver
Syslog messages use UDP and/or TCP
Log messages are sent in cleartext
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Syslog in Unix-like Systems
Syslog is a comprehensive logging system that is used to manage information generated by the kernel and system utilities
It allows messages to be sorted by their sources and routed to various destinations
• Examples:• Log files and user’s terminals
It is controlled through the configuration file /etc/syslog.conf
To log all messages to a file, replace the selector and action fields with the wildcard
Configure Syslog to log all authorize messages with a priority of lower or higher to the /var/log/syslog
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Steps to Set Up a Syslog Server for Unix Systems
1.• Create a central syslog server that accepts incoming syslog messages
2.• Configure to listen on UDP port 514
3.• Run syslogd with –r option
4.• Configure other servers to log their message to this server
5.• Modify the action field in the syslog.conf file as below
• Auth.* @10.0.0.2
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Centralized Syslog Server
• Central Syslog is kept on a different segment for storage security
• Attacker finds it difficult to delete the logs• Log messages allow co-relation of attacks across different
platforms• It has an easier backup policy• Real time alerts are generated by using tools such as Swatch
Advantages of Centralized Syslogging:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Centralized Syslog Server (cont’d)
Routers and Switches Unix/Windows serversFirewall
Central Syslog Server
Log Data Mining
Online Alerting
Log Analysis and Reporting
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
IIS Centralized Binary Logging
Centralized binary logging is a process where multiple Web sites send binary and unformatted log data to a single log file
It is a server property, so all the Web sites on that server are configured to write log data to the central log file
It reduces administration burden for Internet Service Providers (ISPs), and helps in collecting and storing the logged data
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Extended Logging in IIS Server
Enables extended logging in IIS servers
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Time Synchronization
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Why Synchronize Computer Times?
A key component of any computer security system is regular review and analysis of both certain standard system log files as well as the log files created by firewalls and intrusion detection systems
If computers are running on different times, it becomes almost impossible to accurately match actions logged on different computers
In case you suffered an intrusion, though your computers have the same time, it might be difficult to correlate logged activities with outside actions if your computer time is wrong
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
What is NTP?
An Internet standard protocol (built on top of TCP/IP) that assures accurate synchronization to the millisecond of computer clock times in a network of computers
NTP synchronizes client workstation clocks. Running as a continuous background client program on a computer, NTP sends periodic time requests to servers, obtaining server time stamps to adjust the client's clock
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
NTP Stratum Levels
NTP stratum levels define the distance from the reference clock
A reference clock is a stratum-0 device that is assumed to be accurate and has little or no delay associated with it
The reference clock synchronizes to the correct time (UTC) using long wave radio signals, GPS transmissions, CDMA technology or other time signals such as WWV, DCF77, etc.
Stratum-0 servers cannot be used on the network; instead, they are directly connected to computers which then operate as stratum-1 servers
A server that is directly linked to a stratum-0 device is called a stratum-1 server
Higher stratum levels are distanced from the stratum-1 server over a network path
A stratum-2 server gets its time over a network link, via NTP, from a stratum-1 server
A stratum-3 server gets its time over a network link, via NTP, from a stratum-2 server, and so on
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Direct Connection(e.g.. RS 232)
Network Connection
NTP
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
NIST Time Servers
time-a.nist.gov129.6.15.28 NIST, Gaithersburg, Maryland
time-b.nist.gov129.6.15.29 NIST, Gaithersburg, Maryland
time-a.timefreq.bldrdoc.gov132.163.4.101 NIST, Boulder, Colorado
time-b.timefreq.bldrdoc.gov132.163.4.102 NIST, Boulder, Colorado
time-c.timefreq.bldrdoc.gov132.163.4.103 NIST, Boulder, Colorado
utcnist.colorado.edu128.138.140.44University of Colorado, Boulder
time.nist.gov192.43.244.18NCAR, Boulder, Colorado
Time-nw.nist.gov131.107.13.100Microsoft, Redmond, Washington
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
NIST Time Servers (cont’d)
nist1.symmetricom.com69.25.96.13Symmetricom, San Jose, California
nist1-dc.WiTime.net206.246.118.250WiTime, Virginia
nist1-ny.WiTime.net208.184.49.9WiTime, New York City
nist1-sj.WiTime.net64.125.78.85WiTime, San Jose, California
nist1.aol-ca.symmetricom.com207.200.81.113 Symmetricom, AOL facility, Sunnyvale, California
nist1.aol-va.symmetricom.com 64.236.96.53 Symmetricom, AOL facility, Virginia nist1.columbia
countyga.gov 68.216.79.113 Columbia County, Georgia
nist.expertsmi.com 71.13.91.122 Monroe, Michigan
nist.netservicesgroup.com 64.113.32.5 Southfield, Michigan
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Configuring the Windows Time Service
• Click Start, click Run, type regedit, and then click OK• Locate and then click the following registry subkey: HKEY LOCAL
MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters• In the right pane, right-click ReliableTimeSource, and then click Modify• In Edit DWORD Value, type 1 in the Value data box, and then click OK• Locate and then click the following registry subkey: HKEY LOCAL
MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters• In the right pane, right-click LocalNTP, and then click Modify• In Edit DWORD Value, type 1 in the Value data box, and then click OK• Quit Registry Editor• At the command prompt, type the following command to restart the Windows Time
Service, and then press ENTER: net stop w32time && net start w32time• Run the following command on all the computers other than the Time Server to reset the
local computer's time against the Time Server: • w32tm -s
To configure Windows Time Service to use an internal hardware clock, follow these steps:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Event Correlation
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Event Correlation
Event correlation is a procedure, assigned with a new meaning for set of events that occurs in a predefined interval of time
During this process, some events may be added and some events may be deleted
It happens usually inside the log management platform
In general, event correlation process is implemented with the help of simple event correlator software
• Event aggregation• Event masking• Event filtering• Root cause analysis
The four different steps in event correlation:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Types of Event Correlation
• This type of correlation is used when one common OS is used throughout the network in an organization
• Example, organization running Microsoft Windows OS (any version) for all their servers may be required to collect event log entries, do trend analysis diagonally
Same-platform correlation
• This type of correlation is used when different OS and network hardware platforms are used throughout the network in an organization
• Example, clients may use Microsoft Windows, yet they use Linux-based firewall and email gateway
Cross-platform correlation
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Prerequisites for Event Correlation
• Transmitting of data from one security device to other until it reaches a consolidation point in the automated system
• To have a secure transmission and to reduce the risk of exposure during transmission of data, the data has to be encrypted and authenticated
Transmission of data
• After the data is gathered, it must be formatted again from different log formats to single or polymorphic log and that can be easily inserted into the database
Normalization
• After collecting the data, repeated data must be removed so that the data can be correlated more efficiently
• Removing of unnecessary data can be done by compressing the data, deleting repeated data, filtering or combining similar events into a single event and sending that to the correlation engine
Data reduction
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Event Correlation Approaches
• This approach constructs graph with each node as a system components and each edge as a dependency among two components
Graph-based approach
• This approach uses a neural network to detect the anomalies in the event stream, root causes of fault events, etc
Neural network-based approach
• In this approach, events are correlated according to set of rules as followed condition -> action
Rule-based approach
• This approach uses codebook to store set of events and correlate them
Codebook-based approach
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Event Correlation Approaches (cont’d)
• A basic approach where specific events are compared with single or multiple fields in the normalized data
Field-based approach
• This method checks and compares all the fields systematically and intentionally for positive and negative correlation with each other to determine the correlation across one or multiple fields
Automated field correlation
• This approach is used for correlating particular packets with other packets• This approach can make a list of possible new attacks by comparing packets with
attack signatures
Packet parameter/payload correlation for network management
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Event Correlation Approaches (cont’d)
• This method is used to identify whether any system is a relay, a formerly compromised host, and/or to detect the same hacker from different locations
• A series of data sets can be gathered from forensic event data such as, isolated OS fingerprints, isolated port scans, finger information, and banner snatching to compare link attack data to other attacker profiles
Profile/fingerprint-based approach
• This approach is used to map IDS events that targets a particular vulnerable host with the help of a vulnerability scanner
• This approach is also used to deduce an attack on particular host in advance and it prioritizes attack data so that trouble spots can be responded to quickly
Vulnerability-based approach
• The open port correlation approach determines the rate of successful attacks by comparing it with the list of open ports available on the host and that are being attacked
Open-port-based correlation
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Event Correlation Approaches (cont’d)
• This approach is an advanced correlation method which assumes and predicts what a attacker can do next after the attack by studying the statistics and probability and uses only two variables
Bayesian correlation
• This approach eyes the computers' and computer users' behavior and alerts if some anomalous thing is found
Time (clock time) or role-based approach
• This approach is used to extract the attack route information and uses that information to single out other attack data
Route correlation
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Log Capturing and Analysis Tools
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Syslog-ng Logging Systemhttp://www.balabit.com/
• Reliable log transfer• Secure logging using SSL/TLS• IETF syslog protocol standards support• Disk-based message buffering• Flexible message filtering and sorting• Direct database access• Flow control• Heterogeneous environments• Agent for Microsoft Windows platforms• Agent for IBM System i platforms• IPv4 and IPv6 support
Features of Syslog-ng:
Syslog-ng is a flexible and scalable audit trail processing tool for organizations of any size
It provides a centralized, securely stored log of all devices on the network
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Syslog-ng: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
WinSyslog Syslog Serverhttp://www.winsyslog.com/
• Centralized Logging • Interactive Server • Send Syslog Test Message • Standards Compatible • WinSyslog Web Access • Syslog Hierarchy • Email Notifications • Store Messages Persistently • Multiple Instances • Full logging, robust, minimal Resource Usage • Firewall Support • NT Service • Multi-Language Client • Friendly and Customizable User Interface • MWAgent effectively handles for low-memory cases
Features:
WinSyslog is an enhanced syslog server for Windows
It is an integrated, modular and distributed solution for system management
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
WinSyslog: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Kiwi Syslog Serverhttp://www.kiwisyslog.com/
• Display the message in the scrolling window • Log the message to a text file • Forward the message to another syslog server• Log to an ODBC database • Log to the NT Application Event Log • Email the message to someone via SMTP • Triggering a sound alarm • Run an external program• Send an SNMP Trap message • Page someone using NotePager Pro
Syslog messages can then be processed using events such as:
Kiwi Syslog Server receives syslog messages from network devices, and displays them in real time
Actions can be performed on received messages and messages can be filtered by host name, host IP address, priority, message text or time of day
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Kiwi Syslog Server: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Tenable Security Centerhttp://www.nessus.org/
•Quickly rediscover your entire network
Asset Discovery
•Present and make sense of your network security information
Reporting
•Aggregate and Correlate your security logs with the optional LCE module
Log Aggregation and Correlation
•Distribute the scan load throughout your whole network
Distributed Scanning
•Audit the configuration of each system on your network and make sure it matches your local security policy
Configuration Auditing
•Track the action of the network administrators
Security Workflow
Tenable Security Center provides continuous, asset-based security and compliance monitoring
Features:
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Tenable Security Center: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
IISLogger: Development Tool
• Generates additional log information from IIS• Recognizes hacker attacks• Forwards IIS log data to Syslog
It is an addition to the standard Internet Information server logging which:
IISLogger is an ISAPI filter
It is a Dynamic Link Library (.dll) embedded in the IIS environment
Even if the IIS calls an ISAPI filter notification, IISLogger prepares header information and logs this information to syslog in a certain format
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
IISLogger: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Socklog: IDS Log Analysis Tool
• Selects and de-selects the log entries• Minimizes the code size• Provides modular and reliable network logging• Merges different logs and sorts them in order
Benefits of Socklog:
Socklog is a secure replacement tool for Syslog
It is a small, secure and reliable tool
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Microsoft Log Parser: Forensic Analysis Toolhttp://www.microsoft.com/
• Produces the desired information either on the screen, in a file of any desired format into a SQL database
• Allows multiple files to be piped in or out as source or target tables• Generates HTML reports and MS Office objects• Supports conversion between SQL and CSV (Computer System Validation)
Features of Microsoft Log Parser:
It is a command-line program that allows user or administrator to run SQL (Sequential Query Language)-like queries against log files of any format
Output is available from text to XML files and XML files to database storage
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Microsoft Log Parser Architecture
SQL Engine
IIS Logs Text Files Event Log FileSystem
RegistryUser
Plug-in
SYSLOGSQL
DatabaseText FilesScreen,Console
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Microsoft Log Parser: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Firewall Analyzer: Log Analysis Tool
Firewall Analyzer is a web-based firewall monitoring and log analysis tool that collects, analyzes and reports information on enterprise-wide firewalls, proxy servers and radius servers
It helps in tracking intrusion detection, managing user access, auditing traffic and managing network bandwidth efficiently
It uses a built-in syslog server to store the firewall logs and provides comprehensive reports on firewall traffic and security breaches
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Firewall Analyzer Architecture
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Firewall Analyzer: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Adaptive Security Analyzer (ASA) Pro
• Model security specialist expertise• Baseline what is normal for the environment• Identify published threats• Identify activity matching pre-defined criteria• Identify, Measure and Prioritize all anomalous events• Generate root cause insight of threats• Impart new knowledge back into the system
It enables you to:
ASA Pro is a security and threat intelligence application that continuously monitors dynamic, high volume, heterogeneous security-related data, recognizes and quantifies the extent of event abnormality
It provides a flexible mechanism whereby the expert knowledge of the security analyst can be modeled
It reduces the time required to review security-related information
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
ASA Pro Implementation Model
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
ASA Pro: Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
GFI EventsManager
Collects data from all devices that use Windows event logs, W3C, and Syslog and applies the best rules and filtering in the industry to identify key data
This allows you to track when staff swipe their fob, pick up the phone to call home, turn on their PC, what they do on their PC and which files they access during their workday
GFI EventsManager also provides you with real-time alerting when critical events arise and suggests remedial action
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
How Does GFI EventsManager Work?
• GFI EventsManager will automatically collect Windows event logs W3C and Syslog data from remote log sources
Stage 1 – Event Collection
• GFI EventsManager will process collected events and normalize processed events to a central database
Stage 2 – Event processing and centralization
• During this stage, GFI EventsManager will generate meaningful reports on its findings, trigger email, SMS and network alerts on key events and trigger remedial actions such as the execution of a script or executable file on key events
Stage 3 – Generate output/results
GFI EventsManager breaks down the events management process in 3 automated operational stages, making the product easy to use and configure
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
GFI EventsManager
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Activeworx Security Center
Activeworx Security Center is a Security Information and Event Management product
Activeworx Security Center monitors security-related events for a variety of devices from one console
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Activeworx Security Center Desktop
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Ntsyslog
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
EventReporter
Centralized logging tool for Windows
EventReporter processes the NT Event Logs, parses them and forwards the results via Syslog protocol to a central Syslog server
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
EventLog Analyzer
• Event archiving• Automatic alerting• Pre-defined event reports• Historical trending
Features:
EventLog Analyzer is a web-based systems log analysis tool
It collects, analyzes and reports on application, system, security, file server, and DNS server event logs from enterprise-wide Windows and UNIX systems and routers or switches
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
EventLog Analyzer - Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
FLAG - Forensic and Log Analysis GUIhttp://www.dsd.gov.au/
FLAG was designed to simplify the process of log file analysis and forensic investigations
It uses a database as a backend to assist in managing the large volumes of data, this allows flag to remain responsive and expedite data manipulation operations
It is web-based which enables it to be deployed on a central server and shared with a number of users at the same time
Data is loaded into cases which keeps information separated
It also has a system for reporting the findings of the analysis by extensively using bookmarks
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
FLAG Screenshot
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Simple Event Correlator (SEC)http://kodu.neti.ee/
SEC is an open source and platform independent event correlation tool
It accepts input from regular files, named pipes, and standard input, and can thus be employed as an event correlator for any application that is able to write its output events to a file stream
The SEC configuration is stored in text files as rules, each rule specifying an event matching condition, an action list, and optionally a Boolean expression whose truth value decides whether the rule can be applied at a given moment
Regular expressions, Perl subroutines, etc. are used for defining event matching conditions
SEC can produce output events by executing user-specified shell scripts or programs (e.g., snmptrap or mail), by writing messages to pipes or files, and by various other means
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
Summary
Computer security logs contain information on the events occurring within systems and networks
OS logs are most beneficial for identifying or investigating suspicious activity involving a particular host
Syslog allows messages to be sorted by their sources and routed to various destinations
Centralized binary logging reduces administration burden for Internet Service Providers (ISPs), and helps in collecting and storing the logged data
Stratum-0 servers cannot be used on the network; instead, they are directly connected to computers which then operate as stratum-1 servers
Event correlations happen usually inside the log management platform
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited
EC-CouncilCopyright © by EC-Council
All Rights Reserved. Reproduction is Strictly Prohibited