Anomaly based Network Intrusion Detection Systempabitra/facad/06CS6026t.pdf · 2008-05-07 ·...
Transcript of Anomaly based Network Intrusion Detection Systempabitra/facad/06CS6026t.pdf · 2008-05-07 ·...
Anomaly based Network Intrusion Detection System
Dinakara K
Anomaly based Network Intrusion Detection System
Thesis Submitted in Partial fulfillment of the requirements for the Degree
Of
Master of Technology
In
Computer Science and Engineering
By
Dinakara K (06CS6026)
Under the supervision of
Prof. Jayanta Mukhopadhyay
Prof. S.K. Ghosh
Computer Science and Engineering Indian Institute of Technology
Kharagpur -721302, India
(May 2008)
Computer Science and Engineering
INDIAN INSTITUTE OF TECHNOLOGY
KHARAGPUR
Certificate
This is to certify that the thesis entitled “ Anomaly based Network Intrusion Detection System ” which is being submitted to the Indian Institute of Technology,
Kharagpur, for the award of the degree of Master of Technology in Computer Science and
Engineering by Dinakara K ., Roll No. 06CS6026 has been carried out by him under our
guidance. This thesis, in our opinion, is worthy of consideration for the award of degree
of Master of Technology in accordance with the regulations of this institute.
(Dr. Jayanta Mukhopadhyay) (Dr. S. K. Ghosh) Professor, Asst. Professor, Dept. Computer Science and Engineering School of Information Technology Indian Institute of Technology Indian Institute of Technology Kharagpur – 721 302, India Kharagpur – 721 302, India
ACKNOWLEDGEMENTS
Many people deserve to be acknowledged for their contribution to this work and even
more need to be mentioned for their enthusiasm and support in the last one year. This page is
for them all.
I want to start by thanking my project guides Dr. Jayanta Mukhopadhyay and
Dr. S. K. Ghosh. Thanks for their invaluable guidance, incessant inspiration, prolific
encouragement and for just being there whenever I needed you the most. Their untiring help
and constructive suggestions during the course of the project have helped me in learning a lot
and without which it would have been difficult to complete the thesis work.
I express my sincere thanks to Dr. D. K Nanda, Chief Systems Manager, Computer and
Informatics Centre, IIT kharagpur for providing the facility for sniffing the IIT network.
I am deeply indebted to Dr. G Athithan, Head, Intelligence Systems Division, Centre for
Artificial Intelligence and Robotics, Bangalore for his precious guidance and support given for my
thesis work.
Sincere thanks to my friends, Biswajit Paul, Girish Gokuldasan and Dinesh Singh
Kutiyal for their support and constructive suggestions throughout this project as well as the
whole course.
I would love to dedicate this thesis to my parents whose cooperation, support,
affection and well wishes enabled me to complete this endeavour successfully.
Above all I humbly acknowledge the grace and blessings of thy supreme power that
capacitates me to fulfill this well nurtured dream.
Dinakara K (06CS6026)
CONTENTS
ACRONYMS AND ABBREVIATIONS ........................................................................................................ 3 LIST OF FIGURES.................................................................................................................................... 4
LIST OF TABLES ..................................................................................................................................... 6
1. CHAPTER 1 .............................................................................................................7
1.1. INTRODUCTION ................................................................................................................................. 7 1.2. BRIEF HISTORY OF IDS ..................................................................................................................... 7 1.3. TYPES OF IDS ................................................................................................................................... 8 1.4. DETECTION TECHNIQUES................................................................................................................... 9 1.5. DEPLOYMENT SCENARIOS OF IDS .................................................................................................... 11 1.6. SNIFFING THE NETWORK TRAFFIC WITH IDS .................................................................................... 13 1.7. IDS RESPONSES AGAINST ATTACK.................................................................................................... 15 1.8. SNORT, A OPEN SOURCE SIGNATURE BASED IDS ............................................................................. 16 1.9. RELATED WORK ............................................................................................................................. 19 1.10. MOTIVATION AND OBJECTIVE........................................................................................................ 20 1.11. OBJECTIVE.................................................................................................................................... 21 1.12. ORGANIZATION OF THESIS............................................................................................................. 22
2. CHAPTER 2 ...........................................................................................................23
2.1. SYSTEM ARCHITECTURE.................................................................................................................. 23 2.2. SENSOR/DECODER .......................................................................................................................... 23 2.3. PREPROCESSOR ............................................................................................................................... 24 2.4. ANOMALY DETECTION PRE-PROCESSOR ........................................................................................... 25 2.5. DETECTION ENGINE ........................................................................................................................ 26 2.6. ALERT MODULE.............................................................................................................................. 27 2.7. BASIC ANALYSIS AND SECURITY ENGINE (BASE)............................................................................ 28 2.8. OPERATING ENVIRONMENT.............................................................................................................. 30
3. CHAPTER 3 ...........................................................................................................31
3.1. RESEARCH APPROACH..................................................................................................................... 31 3.2. STATISTICAL MOMENTS OR “MEAN AND STANDARD DEVIATION MODEL” ........................................ 36 3.3. HOTELLING’S T2 HYPOTHESIS, A MULTIVARIATE STATISTICAL TECHNIQUE ...................................... 37 3.4. BAYESIAN CLASSIFICATION, A PROBABILISTIC TECHNIQUE............................................................... 38
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 2
4. CHAPTER 4 ...........................................................................................................40
4.1. EXPERIMENTAL RESULTS AND DISCUSSION ...................................................................................... 40 4.2. EVALUATION SCHEME..................................................................................................................... 40 4.3. COMPARATIVE RESULTS .................................................................................................................. 43 4.4. DISCUSSION .................................................................................................................................... 44
5. CHAPTER 5 ...........................................................................................................46
5.1. CONCLUSION .................................................................................................................................. 46
6. APPENDIX A.............................................................................................................50
6.1. CHARTS OF DIFFERENT NETWORK PARAMETERS OBTAINED WHILE EXPERIMENTATION ........................ 50 6.2. SCREENSHOTS OF BASE CONSOLE ................................................................................................... 58 6.3. TYPICAL VALUESOF NETWORK PARAMETERS FOR NORMAL TRAFFIC IN THE TARGET NETWORK .......... 60 6.4. TYPICAL VALUES OF NETWORK PARAMETERS FOR ANOMALOUS TRAFFIC IN THE TARGET NETWORK .. 61
7. APPENDIX B.............................................................................................................62
7.1. GLOSSARY OF TECHNICAL TERMS............................................................................................. 62
8. APPENDIX C.............................................................................................................65
8.1. ATTACK DESCRIPTION ............................................................................................................. 65
9. APPENDIX C.............................................................................................................68
9.1. THE TCP/IP PROTOCOL STACK ................................................................................................ 68 9.2. IP HEADER .............................................................................................................................. 69 9.3. TCP HEADER........................................................................................................................... 70 9.4. UDP HEADER .......................................................................................................................... 71 9.5. ICMP HEADER ........................................................................................................................ 71 9.6. TCP CONNECTION ESTABLISHMENT ......................................................................................... 72 9.7. TCP CONNECTION TERMINATION ............................................................................................. 73
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 3
ACRONYMS AND ABBREVIATIONS
ACL : Access Control List
ARP : Address Resolution Protocol
BASE : Basic Analysis and Security Engine
DDOS : Distributed Denial of Service
DMZ : Demilitarized Zone
DNS : Domain Name Server
DOS : Denial of Service
HTTP : Hyper Text Transfer Protocol
ICMP : Internet Control Message Protocol
IP : Internet Protocol
NIC : Network Interface Card
NIDS : Network Intrusion Detection System
PCRE : Perl Compatible Regular expression
RPC : Remote Procedure Call
SPAN : Switched Port Analyzer
TAP : Test Access Point
TCP : Transmission Control Protocol
TTL : Time to Live
UDP : User Datagram Protocol
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 4
LIST OF FIGURES
FIGURE 1: NETWORK IDS PLACED BEFORE THE GATEWAY FIREWALL................................................. 11
FIGURE 2: NETWORK IDS IN THE DMZ .............................................................................................. 12
FIGURE 3: NETWORK IDS WITHIN THE PRIVATE NETWORK ................................................................. 12
FIGURE 4: NETWORK IDS SNIFFING THE NETWORK IN A HUB ENVIRONMENT..................................... 13
FIGURE 5: NETWORK IDS SNIFFING THE NETWORK USING TAP DEVICE ............................................. 14
FIGURE 6: DEPLOYMENT SCENARIO OF NIDS WITH SENSORS IN STRATEGIC POINTS............................ 15
FIGURE 7: SNIFFED PACKET (SNORT –V).............................................................................................. 17
FIGURE 8: SNIFFED PACKET ( SNORT –DEV)......................................................................................... 17
FIGURE 9: ALERTS GENERATED IN INTRUSION DETECTION MODE........................................................ 18
FIGURE 10: OVERALL SYSTEM ARCHITECTURE...................................................................................... 23
FIGURE 11: NETWORK IDS SENSOR ...................................................................................................... 23
FIGURE 12: NETWORK IDS PRE-PROCESSOR.......................................................................................... 24
FIGURE 13: ANOMALY DETECTION PRE-PROCESSOR ............................................................................. 25
FIGURE 14: SCREENSHOT OF BASE CONSOLE SHOWING THE GENERATED ALERTS ................................ 27
FIGURE 15: BASE CONSOLE SHOWING THE ALERT STATISTICS .............................................................. 28
FIGURE 16: BASE CONSOLE SHOWING THE DETAILS OF SNIFFED PACKET ............................................. 29
FIGURE 17: TIME SLOTS USED IN GENERATING THE NETWORK PROFILE ................................................. 33
FIGURE 18: ALGORITHM FOR GENERATING THE PROFILE ...................................................................... 33
FIGURE 19: ALGORITHM FOR DETECTION............................................................................................. 34
FIGURE 20: FLOW CHART DEPICTING THE OVERALL WORKING OF ANOMALY DETECTION TECHNIQUE 35
FIGURE 21: NORMAL DISTRIBUTION CURVE WITH DIFFERENT CONFIDENCE INTERVALS......................... 36
FIGURE 22: MULTIVARIATE GAUSSIAN DISTRIBUTION CURVE ............................................................... 39
FIGURE 23: TRAFFIC PATTERN IN THE COURSE OF A DAY (MONDAY) .................................................... 50
FIGURE 24: TCP PACKET COUNT IN THE COURSE OF A DAY (MONDAY) ................................................ 50
FIGURE 25: TCP STATISTICS IN THE COURSE OF A DAY ( MONDAY ) ...................................................... 51
FIGURE 26: UDP PACKET COUNT IN THE COURSE OF A DAY (MONDAY) ............................................... 51
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 5
FIGURE 27: UDP STATISTICS IN THE COURSE OF A DAY ( MONDAY ) ..................................................... 52
FIGURE 28: ICMP PACKET COUNT IN THE COURSE OF A DAY ( MONDAY ) ........................................... 52
FIGURE 29: ICMP PACKET COUNT IN THE COURSE OF A DAY ( MONDAY ) ............................................ 53
FIGURE 30: NUMBER OF CONNECTIONS IN THE COURSE OF A DAY (MONDAY)...................................... 53
FIGURE 31: CONNECTION STATISTICS IN THE COURSE OF A DAY (MONDAY)......................................... 54
FIGURE 32: TRAFFIC STATISTICS IN THE COURSE OF A DAY (SATURDAY )............................................... 54
FIGURE 33: TRAFFIC STATISTICS IN THE COURSE OF A DAY ( SUNDAY ).................................................. 55
FIGURE 34: TRAFFIC STATISTICS IN THE COURSE OF A WEEK.................................................................. 55
FIGURE 36: INTRUSIVE TRAFFIC STATISTICS IN THE COURSE OF A DAY (MONDAY) ................................ 56
FIGURE 37: INTRUSIVE TRAFFIC STATISTICS IN THE COURSE OF A WEEK ................................................ 57
FIGURE 38: AVERAGE TRAFFIC STATISTICS IN THE COURSE OF A DAY (MONDAY ) ................................ 57
FIGURE 39: BASE CONSOLE DISPLAYING THE TRAFFIC STATISTICS BY PROTOCOL................................. 58
FIGURE 40: BASE CONSOLE DISPLAYING THE ALERTS STATISTICS ......................................................... 58
FIGURE 41: BASE CONSOLE DISPLAYING UNIQUE ALERTS .................................................................... 59
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 6
LIST OF TABLES
TABLE 1: TYPICAL VALUES OBTAINED FOR THE NORMAL AND INTRUSIVE NETWORK TRAFFIC WITH
HOTELLING’S AND BAYESIAN DISCRIMINATOR FUNCTIONS 42
TABLE 2: CHART SHOWING THE COMPARATIVE RESULTS OF THE EXPERIMENTS 43
TABLE 3. EXPERIMENTAL RESULTS ON MIT_LL DARPA 1999 DATA SET 44
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 7
1. CHAPTER 1
1.1. INTRODUCTION
Internet is forcing organizations into an era of open and trusted
communications. This openness at the same time brings its share of vulnerabilities
and problems such as financial losses, damage to reputation, maintaining
availability of services, protecting the personal and customer data and many more,
pushing both enterprises and service providers to take steps to guard their
valuable data from intruders, hackers and insiders. Intrusion Detection System has
become the fundamental need for the successful content networking.
IDS provide two primary benefits: Visibility and Control [1]. It is the
combination of these two benefits that makes it possible to create and enforce an
enterprise security policy to make the private computer network secure. Visibility
is the ability to see and understand the nature of the traffic on the network while
Control is the ability to affect network traffic including access to the network or
parts thereof. Visibility is paramount to decision making and makes it possible to
create a security policy based on quantifiable, real world data. Control is key to
enforcement and makes it possible to enforce compliance with security policy.
1.2. BRIEF HISTORY OF IDS
The idea of detecting the intrusions or system misuses by looking at some
kind malicious patterns in the network or user activity was initially conceived by
James Anderson in his report titled “Computer Security Threat Monitoring and
Surveillance” [2] to US Air Force in the year 1980.
In the year 1984, the first prototype of Intrusion Detection System which
monitors the user activities, named “Intrusion Detection Expert System” (IDES)
was developed. In the year 1988, “Haystack” became the first IDS to use patterns
and statistical analysis for detecting malicious activities, but it lacked the
capabilities of real time analysis.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 8
Meanwhile, there were other significant advances occurring at University of
California Davis' Lawrence Livermore Laboratories. In the year 1989, they built a
IDS called “Network System Monitor” (NSM) for analyzing the network traffic.
This project was subsequently developed into IDS named “Distributed Intrusion
Detection System” (DIDS). “Stalker” based on DIDS became the first commercially
available IDS and influenced the growth and trends of future IDS. In the Mid 90’s,
SAIC developed “Computer Misuse Detection System” (CMDS), a host based IDS.
US Air Force’s Cryptographic support centre developed “Automated Security
Incident Measurement” (ASIM), which addressed the issues like scalability and
portability.
The intrusion detection market began to gain in popularity and truly
generate revenues around 1997. In that year, the security market leader, ISS,
developed a network intrusion detection system called “Real Secure”. A year later,
Cisco recognized the importance of network intrusion detection and purchased
the Wheel Group, attaining a security solution they could provide to their
customers. Similarly, the first visible host-based intrusion detection company,
Centrax Corporation, emerged as a result of a merger of the development staff
from Haystack Labs and the departure of the CMDS team from SAIC. From there,
the commercial IDS world expanded its market-base and a roller coaster ride of
start-up companies, mergers, and acquisitions ensued.
Martin Roesch, in the year 1998 launched a light weight open source
Network IDS named “SNORT” [3], which has since then gained much popularity.
In year 1999 Okena Systems worked out the first Intrusion Prevention System
(IPS) under the name “Storm Watch”. IPS are the systems which not only detect
the intrusions but also are able to react on alarming situation. These systems can
co-operate with firewall without any intermediary applications.
1.3. TYPES OF IDS
Depending upon the level of analysis IDS is classified into two major types:
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 9
Network based IDS (NIDS):
Monitors and analyzes the individual packets passing around a network for
detecting attacks or malicious activities happening in a network that are designed
to be overlooked by a firewall’s simplistic filtering rules.
Host based IDS (HIDS):
Examines the activity on individual computer or host on which the IDS is
installed. The activities include login attempts, process schedules, system files
integrity checking system call tracing etc. Sometimes two kinds of IDS are
combined together to form a Hybrid IDS.
Generally IDS has two components –
Central Administration (Management) Module:
Provides centralized facility for managing and monitoring of all the
installations of Intrusion Detection System and hence centralized way of analyzing
and detecting the intrusions. It has the complete view of the various activities and
events occurring in different segments of the organizational network. Moreover
the policy settings, actions to be triggered, patches/signature updation, fine
tuning of sensors can be achieved with this module.
IDS Sensors (Agents):
Analyses the network traffic and identifies attacks and security breaches,
which take place by exploiting the technology of network implementation, reports
the alerts to the Management module and performs the preset actions. IDS Agents
are more autonomous in their functions as compared to the Sensors.
1.4. DETECTION TECHNIQUES
Various techniques are in place for intrusion detection which can be broadly
classified as follows.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 10
Signature/pattern based Detection:
In this technique, the sensors which are placed in different LAN segments
filter and analyse network packets in real time and compares them against a
database of known attack signatures. Attack signatures are known methods that
intruders have employed in the past to penetrate a network. If the packet contents
match an attack signature, the IDS can take appropriate countermeasure steps as
enabled by the network security administrator. These countermeasures can take
the form of a wide range of responses. They can include notifications through
simple network management protocol (SNMP) traps or issuance of alerts to an
administrator’s email or phone, shutting down the connection or shutting down
the system under threat etc.
An advantage of misuse detection IDS is that it is not only useful to detect
intrusions, but it will also detect intrusion attempts; a partial signature may
indicate an intrusion attempt. Furthermore, the misuse detection IDS could detect
port scans and other events that possibly precede an intrusion.
Unauthorised Access Detection:
In unauthorised access detection, the IDS detects attempts of any access
violations. It maintains an access control list (ACL) where access control policies
for different users based on IP addresses are stored. User requests are verified
against the ACL to check any violations
Behavioural Anomaly (Heuristic based) Detection:
In behavioural anomaly detection method, the IDS is trained to learn the
normal behavioural pattern of traffic flow in the network over an appropriate
period of time. Then it sets a baseline or normal state of the network’s traffic,
protocols used and typical packet sizes and other relevant parameters of network
traffic. The anomaly detector monitors different network segments to compare
their state to the normal baselines and look for significant deviations.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 11
Protocol Anomaly Detection:
With this technique, anomaly detector alerts administrator of traffic that
does not conform to known protocol standards. As the protocol anomaly detection
analyzes network traffic for deviation from standards rather than searching for
known exploits there is a potential for protocol anomaly to serve as an early
detector for undocumented exploits.
1.5. DEPLOYMENT SCENARIOS OF IDS
There exist three strategic locations where NIDS can be installed in the
network for effective monitoring of the network, as depicted in the diagrams
below.
Before the Gateway firewall:
In this point, the NIDS can keep track of all network events of interests, even
those attacks which subsequently may fail. As it has to handle large traffic, NIDS
ought to be installed on a faster machine so that analysis is done in real time. Also
it has to be configured correctly so that number of false alarms can be reduced.
Figure 1 shows such a configuration.
Figure 1: Network IDS placed before the Gateway Firewall
Internet Private Network
Router Firewall Firewall
DMZ
Network IDS Public Servers
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 12
In the DMZ (De-Militarized Zone):
Placing IDS within the DMZ enables it to monitor the traffic which is already
partly filtered off through the gateway firewall as depicted in figure 2. This
reduces the burden on the IDS but also limits its visibility
Figure 2: Network IDS in the DMZ
Inside the private corporative network:
The last possibility where NIDS can be stationed is within the corporate
network as shown in figure 3. Such a location aims at monitoring the attacks
emerging from the local networks and also those which are transmitted via
firewall. As the number of attacks possible in this place is lesser than the
preceding cases, this makes the application demands smaller. In this case IDS
generates few false alarms. The scope of visibility is limited to within the corporate
network, thus will not be able to detect the failed attacks as in the previous cases.
Figure 3: Network IDS within the private network
Internet Private Network
Router Firewall Firewall
DMZ
Network IDS Public Server
Internet Private Network
Router Firewall Firewall
DMZ Network IDS
Public Servers
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 13
It is always advisable to install NIDS on systems other than firewall so that
attacker using the fact that firewall together with the IDS on a single computer can
pump in malicious traffic to generate too many false alerts, and at the same time
consuming system resources affecting the operations of firewall.
1.6. SNIFFING THE NETWORK TRAFFIC WITH IDS
In order to monitor the network, the traffic in that segment of the network
has to be made available to the Network IDS. There exists several ways to
eavesdrop the network packets without obstructing their normal flow across the
network as mentioned below.
Sniffing the network packets in a Hub environment
Figure 4: Network IDS sniffing the network in a Hub environment
A network Hub is a physical layer device, hence whenever data frames
arrive, it simply broadcasts them to all other ports. Only the destination system
processes the data while other machines discard. In such an environment, IDS can
be connected to one of the Hub ports with its NIC in promiscuous or general
mode which enables it to get all the network packets moving around the network.
Such a configuration is depicted in figure 4.
Eavesdropping via port mirroring or SPAN (Switched Port ANalyser) port in a
switched environment:
In a switched network, the packets from a source machine are forwarded
only to the respective destination machine as specified by the IP address unlike in
Hub
Network IDS
Hub
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 14
the case of a network connected via Hub where packets are broadcasted to every
other machine in the network. In such an environment, sniffing is made possible
by a technique called Port Mirroring or Switched Port Analyzer where the
mirrored port gets a copy of packet from all other ports. Machine with IDS is
connected to the mirrored port or SPAN port in promiscuous mode so that it can
process all the packets irrespective of their destination. Because of the aggregation
of traffic on a single SPAN port, there are chances of packet drop.
Sniffing the traffic using Network TAP (Test Access port):
Figure 5: Network IDS sniffing the network using TAP device
Network TAPs [4] are the hardware devices having three interfaces, entry,
exit and test port. IDS is connected to the test port where it can see the entire
network traffic as shown in figure 5. TAPs does not introduce any delay or affect
the data movement in the network and operates transparently as it doesn’t possess
IP and hardware address.
Stealth mode operation
The Network IDS has to operate transparently to avoid the intruders from
targeting the IDS itself. So generally the IDS is configured to work in a special
mode called “Stealth mode”. In this arrangement, the IDS sniffing interface is put
in promiscuous mode without assigning the IP address, thus only listening to the
packets flowing across the network keeping its presence transparent from network
users.
Usually the IDS has two Network interfaces, one to monitor the network and
the second one for administrative purposes, like configuring IDS, updating
Internet Private Network
Router TAP Switch
Network IDS
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 15
signatures, communication with IDS sensors/Manager ,dispatching alerts etc.
Attacker can easily detect the configuration and location of IDS by analyzing these
messages in the network. It is possible therefore to guard the IDS by encoding its
messages or to create a separate network for management as shown in the
diagram. The advantage of having a separate network between IDS Manager and
IDS Sensors is not only to provide security but also to ensure “out of band”
communication, meaning no bandwidth of the existing network is utilized for its
communication.
Figure 6: Deployment scenario of NIDS with sensors in strategic points
It is generally recommended to use IDS sensors inside and outside the
firewall or between each firewall in a multi-layered environment and host based
IDS on all critical or key hosts. IDS Management Module and its sensors
communicate via zero bandwidth LAN segment in a transparent or stealth
operation mode. This kind arrangement enables the IDS to have complete view of
the organizational network and can even detect the failed attempts of attacks
while reducing the chances of being compromised. Figure 6 depicts a complete
deployment scenario of Network IDS.
1.7. IDS RESPONSES AGAINST ATTACK
Whenever IDS detects any intrusions or attacks, it reacts as per the
preconfigured settings. The responses can range from mere alert notifications to
blocking of the attacks based on the severity. The appropriate reactions on the
Subnet
Internet SubnetRouter Firewall Firewall
DMZ
IDS Sensor Public Server
IDS Sensor IDS Sensor
IDS Sensor
IDS Admin Console
Switch
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 16
threats are a key issue for safety and efficacy. Generally the responses can be of
three types [2]
Active response:
IDS by itself cannot block attacks, however can take such actions which can
lead to stopping of attacks. Such actions can be for example, sending TCP reset
packets to the machine(s) which is being the target of attack, reconfiguring
router/firewall as to block the malicious connection. In extreme cases, IDS can
even block all the network traffic to avoid potential damage to the firm.
Passive response:
Passive solutions deliver information to IDS administrator on the current
situation and leave the decision to take appropriate steps to his discretion. Many
commercial systems rely on this kind of reactions. Examples for this kind of
actions can be simple alarm messages and notifications. Notifications can be sent
on email, cellular phone or via SNMP messages.
Mixed response:
Mixed responses combine both active as well as the passive responses
appropriately as per the needs of situation.
1.8. SNORT, A OPEN SOURCE SIGNATURE BASED IDS
SNORT is a libpcap based lightweight network intrusion detection system,
capable of performing real-time traffic analysis and packet logging on IP networks [5]. It can perform protocol analysis, content searching/matching and can be used
to detect a variety of attacks and probes, such as buffer overflows, stealth port
scans, CGI attacks, OS fingerprinting attempts, and much more. Snort uses a
flexible rules language to describe traffic that it should collect or pass, as well as a
detection engine that utilizes a modular plug-in architecture. Snort has a real-time
alerting capability as well, with alerts being sent to syslog, a separate “alert” file or
even Windows computer via Samba.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 17
The first version of SNORT was released in 1998 by Martin Roesch under
GPL license. Currently version 2.8 is running. Snort has three primary modes of
operation [3]. They are
Sniffer
In this mode, SNORT simply eavesdrop the packets and displays them like
tcpdump program. Depending on the flags used with SNORT, we can determine
how detailed information we want to avail. Figure 7 shows the minimal details of
a packet captured by SNORT.
Figure 7: Sniffed Packet (snort –v)
Packet logger:
Whenever the SNORT user wants to record the packets captured by the IDS,
SNORT has to be run in the Packet logger mode, specifying the directory name
where the packets are to be logged. It logs packets either in tcpdump format
(binary) or in decoded ASCII format. Figure 8 shows descriptions of packets
sniffed by the SNORT program.
Figure 8: Sniffed Packet ( snort –dev)
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 18
Intrusion Detection mode:
In this mode, SNORT will not record every packet that it sniffs but logs only
those events which triggered its rules as shown in figure 9.
Figure 9: Alerts generated in intrusion detection mode
SNORT Rule structure:
SNORT rules are written in PCRE format which are straight forward and
quite powerful. These rules are editable as per the need. Generally the rule
structure has two logical parts
Rule header contains
The type of action SNORT has to take on matching of a rule (e.g. alert, log)
Protocols (IP, ICMP, TCP, UDP)
Sender IP address and the port number
Flow direction (incoming, outgoing or both)
Receiver IP address and the port number
Source port and destination.
Rule options contains
Alert messages and information on which parts of the packet should be inspected
to determine if the rule action should be taken.
Rule Header Rule Option
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 19
The sample SNORT rule given above says that if the payload of a TCP
packet matches with the content “00 01 86 a5” originated from any source address
and any port number to the destination address 192.168.1.0/24 with port number
111 generate alert message “mountd access”.
1.9. RELATED WORK
Network intrusion detection systems like snort [3] or Bro [11] typically use
signature based detection, matching patterns in network traffic to the patterns of
known attacks. This works well, but has the obvious disadvantage of being
vulnerable to novel attacks. An alternative approach is anomaly detection, which
models normal traffic and signals any deviation from this model as suspicious.
The idea is based on work by Forrest et al. (1996), who found that most UNIX
processes make highly predictable sequences of system calls in normal use.
Network anomaly detectors look for unusual traffic rather than unusual
system calls. ADAM (Audit Data and Mining) [12] is an anomaly detector trained
on both attack-free traffic and traffic with labelled attacks. It monitors port
numbers, IP addresses and subnets, and TCP state. ADAM uses a naive Bayes
classifier which means that the probability that a packet belongs to some class
(normal, known attack, or unknown) depends on the a-priori probability of the
class, and the combined probabilities of a large collection of rules under the
assumption that they are independent.
In the IDES/NIDES systems [9], [10], a statistical based anomaly detection
technique is used to represent the expected normal behaviour of a subject and
variance due to noises. The statistical-based anomaly detection technique
overcomes the problems with rule-based anomaly detection technique in handling
noises and variances. However, the statistical technique in IDES/NIDES is a
univariate technique that is applied to only one behaviour measure, where as
many intrusions involve multiple subjects and multiple actions having impact on
multiple behaviour measures. Hence, a multivariate anomaly detection technique
is needed for intrusion detection.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 20
Matthew V. Mahoney and Philip K. Chan developed “Packet Header Anomaly
detection for identifying Hostile Network (PHAD)” [16],[17] that learns the normal
ranges of values for each packet header field at the data link (Ethernet), network
(IP), and transport/control layers (TCP, UDP, ICMP). PHAD detects some of the
attacks in the DARPA data set that involve exploits at the transport layer and
below.
The paper, “Detecting Novel Network Intrusions Using Bayes Estimators” [18]
authored by Daniel Barbara and et al suggests a method called pseudo-Bayes
estimators as a means to estimate the prior and posterior probabilities of new
attacks. Then a Naive Bayes classifier is used to classify the instances into normal
instances, known attacks and new attacks.
1.10. MOTIVATION AND OBJECTIVE
Despite the fact that intrusion detection systems are commercially developed
and used for more than a decade, there still exist many issues around IDS. Some of
the shortcomings of the current IDS which handicap its effectiveness are discussed
below.
a) Only the known attacks are detected in signature based techniques which
simply means no protection is offered against novel attacks or new variants
of existing intrusions. A small variation in the attack pattern can
invalidate a signature. By the time the new signatures/patches come up
the intrusions might have done the intended damages.
b) How well a signature captures the attacks in its string is again a matter of
concern. There are quite a few such poorly written signature codes. So the
actual attack pattern may stretch across multiple packets, easily evading
the detection system.
c) In order to perform an exhaustive signature based search, the processing
and memory needs are very high and in the real time scenario, there is
quite likely hood of missing genuine attacks. Also, there is the
problem of ever increasing attack signature databases.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 21
d) Also the attackers can frame such malicious packets that are likely to have
many attack signatures to keep the detection engine busy and in the course
of action some packets with real attack patterns will find their way into the
internal network, thus evading the detection system
e) There is another class of attacks which targets the detection algorithms as
elucidated below. String matching algorithms are the core component of
any signature detection mechanism and there is not a single string
matching algorithm which can be efficient in any given situation. So the sly
intruders can fabricate and send the packets which cause the algorithms
to run in the worst case complexities.
f) And what if the attacker sends packets with signatures spread across
multiple packets, use techniques like stealth scanning.
g) In anomaly approach, though new kinds of intrusions are detected, this
benefit is paralyzed by high number of false alarms. More over
improper/ insufficient training to anomaly module results in showing the
genuine changes in the network traffic pattern as suspicious activities only
to raise the number of false positives and false negatives.
1.11. OBJECTIVE
The aim of the present work was to design and develop of a Anomaly or
behavioural based Network Intrusion Detection System which can detect
intrusions based on behavioural patterns (i.e. without the use of signatures) and
can also detect novel attacks which are anomalous in nature.
The work also aimed at reducing number of false alarms by characterizing
the target network with appropriate network parameters and analyzing them with
mathematical models.
Literature survey reveals that, the Bayesian Analysis is successfully used in
the SPAM filters but in the area of IDS it is still not explored to great extent. So in
this work, Bayesian classification technique is used for discriminating the
anomalous attacks from that of normal activities. Hotelling’s Multivariate
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 22
statistical hypothesis technique and statistical mean- variances model are also
being used.
The project is integrated with a open source signature based IDS called
SNORT so that it forms a complete package having both signature and anomaly
techniques for effective defence against the Network attacks
1.12. ORGANIZATION OF THESIS
This report is organized as follows. Section 1 gives brief introduction to the
project topic, Types and techniques for IDS, deployment scenarios of IDS etc.
Then related work in the field of IDS is covered. It also talks about the motivation
for taking up the project and objectives set for the project. Chapter 2 deals with the
system architecture, explains the individual components of the IDS. Next section
i.e. Chapter 3 explains the techniques used in the research. Chapter 4 deals with
the results and discussions. Finally chapter 5 covers the conclusion and the future
directions for enhancing the capabilities of the present IDS.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 23
2. CHAPTER 2
2.1. SYSTEM ARCHITECTURE
The proposed architecture of Network IDS has various components as
depicted in the figure 10. This architecture is based on SNORT, which is a open
source Network IDS [19]. The components execute different functionalities which
are discussed below.
Figure 10: Overall System architecture
2.2. SENSOR/DECODER
Figure 11: Network IDS sensor
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 24
The NIC is put in promiscuous mode to sniff all the packets in the network
irrespective of their target. The decoder receives the packets from the libpcap
packet capturing library and processes them. Formal checker evaluates the packet
structure for truncated packet headers and proper checksum, depending on
whether it is an Ethernet, ARP, IP, TCP, UDP or ICMP packets. When Formal
checker detects an error in the packet structure, it informs the decoder and the
packet is discarded from further processing. Figure 11 shows the block diagram of
the sensor/decoder. This module executes following functionalities.
- Sniffs all the network packets visible to it in real time.
- Extract the header and payload information from the Ethernet frame.
- Updates the Ethernet, ARP, RARP, IP, TCP, UDP and ICMP counter
as and when the respective packets are received
- Perform necessary checks on header and payload information.
- Sniffed packets sent to the Pre-processor
2.3. PREPROCESSOR
This module takes the packets from the decoder and performs the functions
like IP de-fragmentation, building the sessions for reassembly of packets etc.
Several pre-processors are available with SNORT to execute the necessary tasks as
depicted in Figure 12. This module also hosts the Anomaly learning and detection
pre-processor used for detecting the intrusions leading to anomalies.
Figure 12: Network IDS pre-processor
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 25
The pre-processor has following responsibilities:
- De-fragments the fragmented IP packets
- Reassembles the TCP packets into streams
- Normalizes Application Layer protocols like Telnet/HTTP
- Detects Port scans/Evasion Attacks
- Pre-processed packets sent to Detection Engine
- Anomaly Detection pre-processor detects the intrusive activities in
the network
2.4. ANOMALY DETECTION PRE-PROCESSOR
This module helps to detect network based intrusions which manifests in
abnormal network behaviour. It runs in two phases, learning (Training) mode and
detection mode. In the learning mode, the module learns the traffic pattern of the
entire network and records the corresponding network parameters. Once the
learning is over, the network profile is generated using the profiler program. This
profile is used to detect the anomalies when the module runs in the detection
mode. Figure 13 shows the structure of Anomaly detection pre-processor.
Figure 13: Anomaly Detection pre-processor
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 26
It performs following functionalities:
In the Learning mode
- Measures the network parameters at regular intervals as configured
by user
- Stores these values into a log file at regular interval
In the Detection mode
- Measures the network parameters at regular intervals
- Reads baselined values from the file
- Finds statistical deviations (Mean and Variance)
- Computes values for Hotelling’s expression and Bayesian
discrimination function
- Triggers the alerts on detecting any abnormalities in the traffic
pattern
2.5. DETECTION ENGINE
It is the main part of the entire system which is responsible for detecting the
attack signatures in the pre-processed packets. The overall system performance
directly depends on this module. Some of the main functions handled by this
module are listed below.
- Parses the rules and build an internal data structure that holds the
rules in a customized tree structure. Once the tree is built, loads it
into memory.
- Passes traffic through this rule tree for comparing the packet header
and data against the rules. (Uses strings matching algorithms)
- Report to Alert module on packets that have found to be carrying
malicious data.
- If any new rules have been added or if existing rules are modified or
deleted then updates the same to the detection engine tree structure.
- When the application is exited this will clean up all memory
allocated for building the detection engine.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 27
2.6. ALERT MODULE
- Sends the alerts triggered by the Detection Engine to Alert Console
in real time.
- Stores the alerts into a alert file (/var/log/snort) and/or into a
Database such as MySQL as per the configuration
Open source php based console, called “Basic Analysis and Security Engine”
(BASE) is integrated with the Alert Module to enhance the user friendliness. The
figure 14 shows screenshot of the BASE console.
Figure 14: Screenshot of BASE console showing the generated alerts
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 28
2.7. BASIC ANALYSIS AND SECURITY ENGINE (BASE)
BASE is a open source code written in the PHP programming language
which displays information from a database in a user friendly web front end [6],[7].
It is based on the code from the “Analysis Console for Intrusion Databases”
(ACID) project. Apache web server has to be setup for running BASE. Figures 15
and 16 shows the screenshots on BASE console
Figure 15: BASE console showing the alert statistics
When used with Snort, BASE reads both tcpdump binary log formats and
Snort alert formats [7]. Snort must be configured to log alerts to the database used
by BASE (for example. MySQL). The alerts from Anomaly detection pre-processor
can also be viewed on BASE console. Once data is logged and processed, BASE
has the ability to graphically display both layer-3 and layer-4 packet information.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 29
It also generates graphs and statistics based on time, sensor, signature, protocol, IP
address, TCP/UDP port, or classification. The BASE search interface can query
based on alert Meta information such as sensor, alert group, signature,
classification, and detection time, as well as packet data such as
source/destination addresses, ports, packet payload, or packet flags.
Thus BASE allows for the easy management of alert data. The administrator
can categorize data into alert groups, delete false positives or previously handled
alerts, and archive and export alert data to an email address for administrative
notification or further processing. Support for user logins and roles, allowing an
administrator to control what is seen through the web interface.
Figure 16: BASE console showing the details of sniffed Packet
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 30
2.8. OPERATING ENVIRONMENT
The development work is carried out in C language on Linux platform to
comply with the SNORT program. The following software/tools are used for the
development and execution of the project
ANJUTA - Open source IDE
BASE - Basic Analysis and Security Engine
GCC - GNU C Compiler to compile the components.
Libpcap - Linux Packet capturing library
MYSQL - Centralized database storage.
RHEL4 - Redhat Enterprise Linux 4
SNORT - Open Source Network Intrusion Detection System
The IDS works efficiently on a system with the following configuration:
Pentium IV 2.0 GHz
512MBRAM
40 GB Hard Disk or higher
10/100 Mbps Ethernet Interface Card.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 31
3. CHAPTER 3
3.1. RESEARCH APPROACH
The primary task was to characterize the target network in terms of suitable
network parameters. The parameters are chosen such that their values will change
perceivably in normal and intrusive conditions. The features considered are the
commonly seen protocols in the network traffic, the traffic data rate and the flow
direction.
In essence, the Anomaly model tries to capture the network behaviour in
terms of two quantities intensity and heterogeneity. Intensity refers to the number of
occurrences of a given network parameter over a period of time (for example
number of TCP connections or number of outgoing HTTP packets etc) while
heterogeneity refers to the observed pattern of the nature of network activities
over time (for example the data rate of HTTP packets in different time segments of
the day or observations like web traffic is more during the beginning of office
hours and then drops. It rises again during the closing hours etc). These two
quantities closely relate to activities occurring in any given network and thus can
represent the behaviour of network under the assumption that network behaviour
has certain degree of repeatability.
Once the network behaviour is quantified with these parameters, the next
step would be to observe how they vary with time. The observation has to be
made on different days of a week because the network behaviour changes over
working days and non working days of a week and also on general holidays. The
Anomaly based IDS has two operational modes.
Learning (or training) mode:
In this mode, the IDS learns the normal traffic behaviour in terms of
representative feature set characterizing the target network. It collects the statistics
of the selected network parameters for different types of days (Week days from
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 32
Monday to Friday, Saturdays and Sundays) and then stores them into a specified
file for subsequent processing. The frequency of statistics collection is set as per
requirement; it is set by default to 10 minutes. IDS is put in this mode for
sufficient period to learn the normal network behaviour. Sufficient training period
is the key factor in reducing the false alarms. When IDS is learning the normal
behaviour, the target network is assumed to be free from attacks and intrusions
Following attributes are considered for characterizing the network:
TCP Packet count (incoming, outgoing and within LAN)
UDP Packet count (-----------------’ ’--------------------)
ICMP Traffic (-----------------’ ’--------------------)
The number of TCP connections
Web Traffic (incoming, outgoing)
DNS Traffic (---------’ ’------------)
Data rates TCP traffic in kb/s (---------’ ’------------)
Data rates UDP traffic in kb/s (---------’ ’------------)
Data rates HTTP traffic in kb/s (---------’ ’------------)
Data rates DNS traffic in kb/s (---------’ ’------------)
Once the learning is over, profile for the target network is generated with the
gathered data using a profiler. If statistics collections is done at every 10 minutes
and the learning period is say 1 month, total 24 sample values are available for
each network parameter corresponding to each hour of the week day. Hence the
profile is generated for each hour of the day over entire week. This implies that
total 168 baseline vectors are established for the entire week, each vector
containing 25 network parameters. The profile also contains 168 inverse matrices
each of the order 25 x 25, accounting for number of parameters in consideration.
This profile is used by Anomaly detection module during the detection phase. The
IDS is also trained to learn the network behaviour in the presence of network
intrusions. Intrusions are simulated using the MIT-DARPA training data set.
Network profile is also generated for this condition. Figure 17 shows the time
slots used for generating the profile.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 33
Figure 17: Time slots used in generating the network profile
When the network environment changes for genuine reasons, it may result
into a number of false positives. In such situations the Anomaly model can be
updated by rerunning the training phase on the changed traffic and rebuilding the
profile using profiler program.
The logic for profile generation is given in figure 18.
Input : The file containing the features values logged during the learning phase
Output : files containing the mean, standard deviations and inverse matrices of
feature set
begin for i =1 to Num .of week days do
for j =1 to Num. of hours in a day do
Read the feature values logged during learning phase;
for k =1 to Num. of network features do
find sum of the values corresponding to the same hour and day of the week;
Compute Average values and standard deviation for each feature;
Compute ∑ −−=
n
ml
T
ml xx1,
))(( μμ where n is the total number of features
Compute the Determinant of above covariance matrices
if Determinant ≤ 0
Consider the neighbouring covariance matrix having positive Determinant
Compute inverse matrix corresponding to each Covariance matrix
end
Figure 18: Algorithm for generating the profile
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 34
Detection mode:
In this mode, IDS detects in real time, the network based attacks leading to
abnormal traffic pattern. The abnormality is decided on the basis of the network
profile constructed earlier. The profile contains 168 vectors corresponding to each
hour of the day over entire week, each vector containing as set of 25 features
which describes the network. The Anomaly detection module samples the selected
network parameters at regular intervals, as in the case of learning mode, checks
whether they comply with already established network profile for that particular
hour and day of the week. If it detects significant deviations, then it triggers alerts.
The logic for detection is given in figure 19
Input : The file containing the network profile
Output : Sends alert in case a event is detected as intrusion
begin for i =1 to Num .of week days do
for j =1 to Num. of hours in a day do
for k =1 to Num. of network features do
Read Average values and standard deviation for each feature;
Read the inverse matrices
Read the determinant matrix corresponding to each inverse matrix
Compute σ)(μ ± for each parameter
if σ)μxσ( μ +>>−
x is intrusive
Compute TX)S(XT )(12 μμ −−= −
if 2T exceeds the threshold flag alerts
Compute )(ln)()(21||ln
21)( 1 IpXSXSXg T
i +−−−−= − μμ
if )( Xg i exceeds the threshold flag alerts
end
Figure 19: Algorithm for Anomaly Detection
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 35
The flow chart in figure 20 shows the overall working of Anomaly Detection
technique.
Figure 20: Flow chart depicting the overall working of Anomaly Detection
Technique
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 36
3.2. STATISTICAL MOMENTS OR “MEAN AND STANDARD
DEVIATION MODEL”
Statistical based anomaly detection techniques use statistical properties
(mean and variance) of normal activities to build a statistical normal profile and
employ statistical tests to determine whether observed activities deviate
significantly from the normal profile [20].
Figure 21: normal distribution curve with different confidence intervals
The arithmetic average, or the mean, is a statistic that measures the central
tendency of a set of data. It is given by,
n
n
i ixμ
∑== 1 Where μ = mean
ix = value of ith observation of a given parameter, i =1… n
n= total number of observation in a sample
The Standard Deviation is a measure of the amount of data dispersion around the
mean. It is given by,
11
)( 2
−
∑=
−=
n
n
i ix μσ Where σ =standard deviation
ix = value of ith observation of a given parameter, i =1… n
μ = mean
n = total number of observation in a sample
The values of μ and σ are established for each of the network parameter ix .
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 37
If the value of ix goes beyond ( σμ *n± ), it simply indicates an anomalous
situation and can be flagged as alert.
It is difficult to determine thresholds above which an anomaly should be
considered intrusive. Setting threshold too low results in false positives and
setting it too high results in false negatives. So the confidence interval is chosen
suitably based on the experimentation [21]. Figure 21 shows different confidence
intervals for a Gaussian distribution.
3.3. HOTELLING’S T2 HYPOTHESIS, A MULTIVARIATE
STATISTICAL TECHNIQUE
When there are enough computational resources and the security level is
also high then "multivariate models" are a good choice since they produce better
results with less false alarm rate as compared to mean and standard deviation
model. Hence these are recommended for the IDS.
Hotelling’s T2 test is a multivariate statistical process control technique that
detects anomalies in the activities of a network. It can be assumed as the
multivariate extension of mean/standard deviation model, employing an n
dimensional mean vector and the corresponding covariance matrix.
Hotelling’s 2Τ statistic for an observation iX is determined by [13],[14]
)()( 12 μμ −−=Τ −i
Ti XSX
Where
).......,,( 321 ipiiii XXXXX = , denotes an observations of p variables at time t
),.......,,( 321 pμμμμμ = , denotes a vector of mean values of p variables at time t
and S is the covariance matrix given by,
∑ −−−
=n
Tii XX
nS
1))((
)1(1 μμ , where n is the data sample size
The computed 2Τ value is small if the data point conforms to the norm
profile. If the value of the 2Τ statistic is greater than a threshold value, then the
null hypothesis that the event is normal is rejected and signals anomalous
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 38
behaviour. The threshold value is set based on the observed values of 2Τ for
normal and intrusions during the learning phase. Hotellings 2Τ test provides a
complete data model of multivariate data. Since it uses the covariance matrix S of
p variables, it detects both mean shifts and their interrelationship in a multivariate
manner which is important in finding the network anomalies. The test detects
three kinds of events. They are normal, suspicious and Intrusive. Normal
corresponds to the events which comply with previous normal traffic pattern.
Suspicious means the events which are deviated to some extent from their normal
behaviour and Attack indicates there is a large variation in the observed and
expected traffic pattern.
3.4. BAYESIAN CLASSIFICATION, A PROBABILISTIC
TECHNIQUE
In probabilistic classification method, a pattern is assigned to the class that is most
probable, given the observed features, i.e., point x of a feature space is assigned to
the class that maximize )/( xCp j
The classification problem is formulated in terms of estimating the posterior
probability that pattern x belongs to one of the m data classes
Posterior probability depends on
- The prior probability )( iCp i.e. the likelihood that a random selected pattern
belong to class iC
- The class conditional probability density function )/( iCxp i.e. the distribution of
patterns of class ic in the selected space.
Baye’s Theorem:
Bayesian statistics, in the most general form, provides a framework for
combining observed data with prior assumptions in order to model stochastic
systems [23], [24].
)()()./(
)/()()(
)/()/(
1
iM
iii
ii
ii Cp
CpCxp
CxpCpxpCxpxCp
∑=
==
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 39
Any function that computes the conditional probabilities )/( xCp i is referred
to as discriminant function. Given an observation x , the Bayes theorem provides a
method to compute )/( xCp i .
)(xp can be ignored, since it is the same for all the classes and thus does not help in
discriminating the classes.
The likelihood function )/( iCxp denotes a probability density function of
the vector samples x given a particular estimate iC of the underlying probability
distribution generating that data. A multivariate normal distribution is assumed
for )/( iCxp . Figure 22 shows the multivariate Gaussian distribution curve.
Figure 22: Multivariate Gaussian distribution curve
A Gaussian or multivariate normal distribution is characterized by its mean value
vector μ and its covariance matrix S and has the distribution function,
)}()(exp{||||)2(
1),( 121
2/1μμ
πμ −−−=Σ − XSX
Sf T
p ---------- (2)
Here X is a p dimensional pattern vector of real valued attributes
The discriminant function )(Xgi can be derived by using the equations (1) and (2).
)(ln)()(21||ln
21)( 1 IpXSXSXg T
i +−−−−= − μμ
The values of )(Xgi can distinguish the intrusions from the normal events.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 40
4. CHAPTER 4
4.1. EXPERIMENTAL RESULTS AND DISCUSSION
To evaluate the system, two major indicators of performances are chosen.
- Detection rate
- False positive rate
Detection rate is defined as the number of intrusion instances detected by the
system divided by the total number of intrusion instances present in the test set.
The false positive rate is defined as the total number of instances that were
wrongly detected as intrusions divided by the total number of normal instances.
These are good measures of performances since they measure what percentage of
intrusions the system is able to detect and how many incorrect classifications it
makes in the process. The following sub sections give the details of evaluation
scheme and the results obtained.
4.2. EVALUATION SCHEME
The Anomaly IDS is trained for five weeks to learn the normal network
traffic of the IIT, Kharagpur. The model considers a vector of 25 network attributes
to describe the target network. The IDS is also trained for more than three weeks
to learn the network behaviour under intrusions. The intrusions are simulated in
the network using MIT-DARPA 1999 data set. The training data contains a total of
4396 vector data points for normal traffic and 2120 vector data points for intrusive
traffic. The training period covers different types week days (working, Saturday
and non working days). The network profile is generated using the training data
which contains a total of 168 vector data points corresponding to each hour of the
day over the entire week. The same training data and the test data is used with all
the three techniques discussed earlier.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 41
About MIT-DARPA IDS Evaluation
In 1998, the Information Systems Technology Group of Lincoln Laboratory at
MIT, in conjunction with the Air Force Research Laboratory (AFRL) and the
Defence Advanced Research Projects Agency (DARPA), began work to develop a
standard for the evaluation of Network IDS. Developing this evaluation meant
the creation of consistent and repeatable network traffic. The traffic was created
through the study of 4 months of data from Hanscom Air Force Base and
approximately 50 other bases. Using that data, they were able to generate and
simulate network traffic, while introducing attacks, probes and intrusions into the
data. Both training and testing data were simulated and two types of traffic were
published. Training data is traffic in which the attacks were known from the start.
A second set of data contains traffic in which the attacks were not described
explicitly. Data sets of Week 1 and Week 3 contain attack free traffic while Week 2
contains training data with attacks. Week 4 and Week 5 are the testing data
containing network attacks in the midst of normal background data. Test Data
sets contains four categories of simulated attacks
DoS – Denial of service (e.g. SYN flood)
R2L -- unauthorized access from remote machine (password guessing)
U2R –unauthorized access to super user or root functions (buffer overflow attacks)
Probing --surveillance and other probing vulnerabilities (port scanning)
A more complete discussion on this is available at the Lincoln Laboratory/ MIT
site [22].
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 42
The table 1 gives the values obtained for the Hotelling’s multivariate expression
and Bayesian Classifier for normal and intrusive network traffic.
Values for Hotelling’s Statistic
Values for Bayesian Classifier
Normal
Intrusive
Normal
Intrusive
1 7.74E+09 1.32E+17 3.07E+08 6.59E+16 2 7.60E+08 9.07E+16 1.48E+07 4.54E+16 3 5.60E+08 6.26E+16 1.32E+07 3.13E+16 4 4.49E+08 6.05E+16 1.07E+07 3.02E+16 5 1.59E+08 4.35E+16 1.04E+07 2.18E+16 6 8.84E+07 2.97E+16 1.03E+07 1.48E+16 7 5.10E+07 2.60E+16 6.70E+06 1.30E+16 8 4.50E+07 2.37E+16 6.52E+06 1.19E+16 9 2.95E+07 1.95E+16 2.88E+06 9.77E+15 10 2.46E+07 1.57E+16 2.74E+06 7.85E+15 11 2.09E+07 1.09E+16 1.71E+06 5.44E+15 12 1.93E+07 9.58E+15 2.16E+05 4.79E+15 13 1.36E+07 9.34E+15 2.60E+05 4.67E+15 14 1.34E+07 6.34E+15 7.19E+05 3.17E+15 15 1.17E+07 5.19E+15 1.29E+06 2.59E+15 16 8.36E+06 5.12E+15 1.40E+06 2.56E+15 17 7.88E+06 3.79E+15 1.41E+06 1.89E+15 18 6.27E+06 2.64E+15 1.59E+06 1.32E+15 19 5.67E+06 2.29E+15 1.63E+06 1.15E+15 20 4.85E+06 2.28E+15 2.42E+06 1.14E+15 21 3.26E+06 3.32E+14 2.84E+06 1.66E+14 22 3.18E+06 2.67E+14 3.13E+06 1.34E+14 23 2.82E+06 2.67E+14 3.94E+06 1.33E+14 24 2.80E+06 2.12E+14 4.18E+06 1.06E+14 25 2.59E+06 1.65E+14 5.85E+06 8.25E+13 26 1.44E+06 1.08E+14 6.70E+06 5.39E+13 27 5.20E+05 7.73E+13 6.82E+06 3.87E+13
Table 1: Typical values obtained for the normal and intrusive network traffic with Hotelling’s and Bayesian discriminator functions By manually analysing a large set of values obtained for Hotelling’s and
Bayesian discriminators, it is found that following values more closely
discriminate the normal activities from the intrusive ones.
Hotellings Technique: On an average, the values for normal activities lie between
1.00E+06 to 5.00E+07 while for intrusive the values are above .90E+08.
Bayesian Technique: On an average, the values for normal activities lie between
2.00E+05 to 9.00E+07 while for intrusive the values are above 1.50E+08
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 43
4.3. COMPARATIVE RESULTS
Attack Name Tools/Data set used Count Detection using different Techniques
Probabilistic (Bayesian Classifier)
Statistical (Hotelliing's Hypothesis)
Statistical (Mean ± 2*SD)
ping flood ping tool 15 15 15 15
DoS attack ddos open source tool 5 5 5 5
TCP RST attack neti open source code 5 5 5 5
TCP Syn flood attack
neti open source code 7 7 7 6
UDP attack neti open source code 10 10 10 10
X mas scan nmap tool 5 5 4 4 NTinfoscan
MIT_ DARPA 1999 Data set
1
0
0
0
pod " " 2 2 2 2
back '' " 2 0 0 0
httptunnel " " 2 0 0 0
land " " 2 2 2 2
secret " " 3 0 0 0
portsweep " " 3 3 3 2
eject " " 3 0 0 0
mailbomb " " 2 2 2 2
ipsweep " " 3 3 2 2
satan " " 2 1 1 1
neptune " " 2 2 2 2
Total 74 62 60 58Detection Accuracy (%) 83.78 81.08 78.38Total Alerts generated 65 64 67No. of Attacks missed 12 16 20
False Positive 4.62 6.25 13.43
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 44
rate (%)
False Negative rate (%) 16.22 21.62 27.03Positive Prediction rate (%) 95.40 90.63 78.30
Table 2: Chart showing the comparative results of the experiments
Table 2. given below shows the results obtained by Daniel Barbara et al using
pseudo-Bayes estimators [6]
Table 3. Experimental results on MIT_LL DARPA 1999 Data set. Source: http://www.cs.ubc.ca/local/reading/proceedings/siam_datamining2001/pdf/sdm01_29.pdf
4.4. DISCUSSION
The experiment clearly revealed that the Bayesian classification method
gives better detection rate and less false positives in detecting the intrusions
among the three techniques discussed in the project. The detection accuracy of
≈ 84 % is achieved using the Bayesian method with the false positive rate of 4.6%.
Hotelling’s statistical method gave a hit rate of ≈ 81% at 6.2% false positive rate.
The performance metrics for statistical Moments (mean and standard deviation)
model yielded hit rate of ≈ 78% while the false positive rate was 13%. The
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 45
comparative analysis with the previous works also reveals that the Bayesian
approach is a superior technique.
In summary, the results show that the approach followed in this thesis is
quite effective and efficient for detecting the network based attacks. It is also
observed that the multivariate statistical techniques are more effective than the
univariate technique, particularly the Bayesian techniques has promising potential
in the future IDS research
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 46
5. CHAPTER 5
5.1. CONCLUSION
Network Intrusion Detection System has a major role to play in safeguarding
the network resources against various kinds of attacks. With the advent of new
vulnerabilities and sophistications in the nature of attacks, new techniques for
intrusion detection have evolved. The main objectives of the research being
increasing the detection accuracy while keeping the false positive rate low.
As stated earlier, the signature based techniques are good but has the
obvious short comings like failure to detect novel attacks, increasing signature
database etc. So the viable alternative would be to analyse the behaviour of the
network as a whole and trying to build the model based on the observations. So
Anomaly based detection has been a wide area of interest for researchers since it
provides the base line for developing promising techniques.
The Anomaly based detection complements the Signature based technique
and helps in identifying the novel attacks which lead to the anomalies in the
network traffic. The major concerns in this method are identifying the appropriate
network features to characterize the network and build a behavioural model and
also the rate of false positives may increase sharply if the IDS is not trained
sufficiently in the target network.
In the present framework of project, discussed the design and development
of “Anomaly based intrusion Detection system” which is built on top of a existing
open source signature based network IDS, called SNORT so to have both the
analysis techniques in a single package .
The Anomaly based component of IDS is trained in the Computer and
Informatics Centre of Indian Institute of Technology (IIT), Kharagpur where the
IIT network traffic is sniffed using a port mirrored switch at the gateway. The IDS
is trained for more than a month in the IIT network at computer and Informatics
centre, to learn the normal traffic pattern. Also it is exposed to the intrusive traffic
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 47
for more than 3 weeks, in a simulated environment by replaying the MIT DARPA
Intrusion Detection System training datasets (1999).
The thesis presented three techniques for detecting anomaly based intrusions
at the network level. Statistical based anomaly detection techniques use statistical
properties and statistical tests to determine whether "observed behaviour" deviate
significantly from the "expected behaviour". The first technique is based on
univariate statistic model with mean and variance. The second method uses the
multivariate Hotelling’s method while the last technique uses the Bayesian
classification technique for discriminating attacks from that of normal activities.
All the three techniques are evaluated with the DARPA IDS evaluation Data
sets (1999) and the results are compared. Bayesian approach proved to be a better
solution than the Hotelling’s Multivariate technique and the method of Statistical
Moments.
Presently, the work caters only to identify and classify the events into
normal and attack classes. It can be extended to detect and classify the attacks into
multiple attack classes. Dynamic updation of the Anomaly Model using Bayesian
Network can also be considered for future enhancement. Different Analysis
techniques like HMM and Fuzzy Logic can also be tried as alternative techniques
for anomaly detection.
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 48
BIBLIOGRAPHY
[1]. R.Coolen, “Intrusion Detection: Generics and State of the Art”, RTO Technical Report 49, http://www.tno.nl/instit/fel/div2/resources/rto-tr-049-ids.pdf
[2]. J. P. Anderson, “Computer Security Threat Monitoring and Surveillance”, Technical Report
April 1980, http://csrc.nist.gov/publications/history/ande80.pdf
[3]. Martin Roesch : “Snort Documents”, http://www.snort.org/docs/
[4]. Net Optics, Inc. “White Paper: Deploying Network Taps with Intrusion Detection Systems”, http://www.netoptics.com/products/downloads.asp?PageID=150&Section=res
[5]. Jack Koziol, “Intrusion Detection with Snort”, Pearson publications, 2003
[6]. Basic Analysis and Security Engine project, http://base.secureideas.net/
[7]. White papers on “Basic Analysis and Security Engine”(BASE), http://whitepapers.techrepublic.com.com/abstract.aspx?docid=266711
[8]. Q. Zhao, J. Sun, S. Zhang, “A hybrid and hierarchical NIDS paradigm utilizing naïve Bayes
classifier”, Canadian conference on Electrical and Computer Engineering, 2004, http://ieeexplore.ieee.org/iel5/9317/29618/01344977.pdf?tp=&isnumber=&arnumber=1344977
[9]. Javitz HS, Valdes A. “The NIDES statistical component description of justification”
Technical Report A010, SRI International, Menlo Park, CA, March 1994. http://www.cs.ucdavis.edu/~wu/ecs236/papers/hw2_NIDES-STA-description.pdf
[10]. Javitz HS, Valdes A. “The SRI statistical anomaly detector”, Proceedings of the 1991
IEEE Symposium on Research in Security and Privacy, May 1991 http://ieeexplore.ieee.org/iel2/349/3628/00130799.pdf?tp=&isnumber=&arnumber=130799
[11]. V. Paxson, “Bro: A System for Detecting Network Intruders in Real-Time”, Computer
Networks, 1999, http://bro-ids.org/publications.html
[12]. D. Barbar´a and S. Jajodia and N. Wu and B. Speegle , “The ADAM project”, http://www.isse.gmu.edu/dbarbara/adam.html
[13]. Nong Ye and Qiang Chen, “An anomaly detection technique based on a chi-square statistic
for detecting intrusions into information systems”, Quality and Reliability Engineering
International, 17:105--112, 2001, http://citeseer.ist.psu.edu/ye01anomaly.html
[14]. Ye, N., Li, X., Chen, Q., Emran, S. M., and Xu, M. “Probabilistic Techniques for Intrusion
Detection Based on Computer Audit Data”, IEEE Transactions on Systems, Man and
Cybernetics, vol.31(4), pp.266--274, July 2001., http://ieeexplore.ieee.org/iel5/3468/20237/00935043.pdf?tp=&isnumber=&arnumber=935043
[15]. A. Qayyum, M. H. Islam, and M. Jamil, “Taxonomy of Statistical Based Anomaly Detection
Techniques for Intrusion Detection”, IEEE International Conference on Emerging
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 49
Technologies, September 17-18,2005 http://ieeexplore.ieee.org/iel5/10430/33125/01558893.pdf?tp=&isnumber=&arnumber=1558893
[16]. M. Mahoney and P. Chan, “PHAD: Packet header anomaly detection for identifying hostile
network traffic”, Technical report, Florida Tech., technical report CS-2001-4, April
2001, http://citeseer.ist.psu.edu/mahoney01phad.html
[17]. Mahoney M. and P. Chan, “Learning models of network traffic for detecting novel attacks",
Technical report, Florida Tech 2002, http://cs.fit.edu/~mmahoney/paper5.pdf
[18]. D. Barbara, N. Wu and S. Jajodia, “Detecting Novel Network Intrusions using Bayes
Estimators”, Proceedings of the 1st SIAM International Conference on Data Mining,
2001, http://www.cs.ubc.ca/local/reading/proceedings/siam_datamining2001/pdf/sdm0129.pdf
[19]. Jack Koziol, “Intrusion Detection with Snort”, Pearson publications, 2003
[20]. R. Dan Reid & Nada R. Sanders, “Operations Management”, 3rd edition., Wiley ,2007
[21]. P. Cisar, S. M Cisar, “Quality Control in Function of Statistical Anomaly Detection in Intrusion
Detection Systems”, SISY 2006 - 4th Serbian-Hungarian Joint Symposium on Intelligent
Systems, www.bmf.hu/conferences/sisy2006/19_Cisar.pdf
[22]. DARPA Intrusion Detection Evaluation, Data Sets and Documentation, 1999 http://www.ll.mit.edu/mission/communications/ist/corpora/ideval/docs/detections_1999.html
[23]. Giorgio Giacinto, Fabio Roli, Luca Didaci, ”Fusion of multiple classifiers for intrusion
detection in computer networks”. Pattern Recognition Letters 24(12): 1795-1803 (2003) http://www.diee.unica.it/informatica/en/publications/papers-prag/IDS-Journal-01.pdf
[24]. R. Puttini, Z. Marrakchi, and L. Me. “Bayesian Classification Model for Real Time Intrusion
Detection”, in 22th International Workshop on Bayesian Inference and Maximum
Entropy Methods in Science and Engineering, 2002. http://www.rennes.supelec.fr/ren/rd/ssir/publis/maxent02_puttini_marrakchi_me.pdf
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 50
6. APPENDIX A
6.1. CHARTS OF DIFFERENT NETWORK PARAMETERS OBTAINED
WHILE EXPERIMENTATION
0
500
1000
1500
2000
2500
3000
3500
21-0
1-08
00:
10
21-0
1-08
00:
50
21-0
1-08
01:
30
21-0
1-08
02:
10
21-0
1-08
02:
50
21-0
1-08
03:
30
21-0
1-08
04:
10
21-0
1-08
04:
50
21-0
1-08
05:
30
21-0
1-08
06:
10
21-0
1-08
06:
50
21-0
1-08
07:
30
21-0
1-08
08:
10
21-0
1-08
08:
50
21-0
1-08
09:
30
21-0
1-08
10:
10
21-0
1-08
10:
50
21-0
1-08
11:
30
21-0
1-08
12:
10
21-0
1-08
12:
50
21-0
1-08
13:
30
21-0
1-08
14:
10
21-0
1-08
14:
50
21-0
1-08
15:
30
21-0
1-08
16:
10
21-0
1-08
16:
50
21-0
1-08
17:
30
21-0
1-08
18:
10
21-0
1-08
18:
50
21-0
1-08
19:
30
21-0
1-08
20:
10
21-0
1-08
20:
50
21-0
1-08
21:
30
21-0
1-08
22:
10
21-0
1-08
22:
50
21-0
1-08
23:
30
Time
Pack
et c
ount
Figure 23: Traffic pattern in the course of a day (Monday)
0
500
1000
1500
2000
2500
3000
3500
21-0
1-08
00:
00
21-0
1-08
00:
40
21-0
1-08
01:
20
21-0
1-08
02:
00
21-0
1-08
02:
40
21-0
1-08
03:
20
21-0
1-08
04:
00
21-0
1-08
04:
40
21-0
1-08
05:
20
21-0
1-08
06:
00
21-0
1-08
06:
40
21-0
1-08
07:
20
21-0
1-08
08:
00
21-0
1-08
08:
40
21-0
1-08
09:
20
21-0
1-08
10:
00
21-0
1-08
10:
40
21-0
1-08
11:
20
21-0
1-08
12:
00
21-0
1-08
12:
40
21-0
1-08
13:
20
21-0
1-08
14:
00
21-0
1-08
14:
40
21-0
1-08
15:
20
21-0
1-08
16:
00
21-0
1-08
16:
40
21-0
1-08
17:
20
21-0
1-08
18:
00
21-0
1-08
18:
40
21-0
1-08
19:
20
21-0
1-08
20:
00
21-0
1-08
20:
40
21-0
1-08
21:
20
21-0
1-08
22:
00
21-0
1-08
22:
40
21-0
1-08
23:
20
Time
TCP
Pack
ets
Figure 24: TCP packet count in the course of a day (Monday)
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 51
0
500
1000
1500
2000
2500
3000
3500
21-01
-08 00
:00
21-01
-08 01
:10
21-01
-08 02
:20
21-01
-08 03
:30
21-01
-08 04
:40
21-01
-08 05
:50
21-01
-08 07
:00
21-01
-08 08
:10
21-01
-08 09
:20
21-01
-08 10
:30
21-01
-08 11
:40
21-01
-08 12
:50
21-01
-08 14
:00
21-01
-08 15
:10
21-01
-08 16
:20
21-01
-08 17
:30
21-01
-08 18
:40
21-01
-08 19
:50
21-01
-08 21
:00
21-01
-08 22
:10
21-01
-08 23
:20
Time
TCP
Pack
et c
ount
TotalTCPpackets
TCPpacketssent
TCPpacketsreceived
TCPPacketsin LAN
Figure 25: TCP statistics in the course of a day ( Monday )
0
20
40
60
80
100
120
140
160
180
21-01
-08 00
:00
21-01
-08 01
:00
21-01
-08 02
:00
21-01
-08 03
:00
21-01
-08 04
:00
21-01
-08 05
:00
21-01
-08 06
:00
21-01
-08 07
:00
21-01
-08 08
:00
21-01
-08 09
:00
21-01
-08 10
:00
21-01
-08 11
:00
21-01
-08 12
:00
21-01
-08 13
:00
21-01
-08 14
:00
21-01
-08 15
:00
21-01
-08 16
:00
21-01
-08 17
:00
21-01
-08 18
:00
21-01
-08 19
:00
21-01
-08 20
:00
21-01
-08 21
:00
21-01
-08 22
:00
21-01
-08 23
:00
Time
UD
P Pa
cket
s
Figure 26: UDP packet count in the course of a day (Monday)
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 52
0
20
40
60
80
100
120
140
160
180
21-01
-08 00
:00
21-01
-08 01
:20
21-01
-08 02
:40
21-01
-08 04
:00
21-01
-08 05
:20
21-01
-08 06
:40
21-01
-08 08
:00
21-01
-08 09
:20
21-01
-08 10
:40
21-01
-08 12
:00
21-01
-08 13
:20
21-01
-08 14
:40
21-01
-08 16
:00
21-01
-08 17
:20
21-01
-08 18
:40
21-01
-08 20
:00
21-01
-08 21
:20
21-01
-08 22
:40
Time
UD
P Pa
cket
cou
ntTotalUDPPackets
UDPPacketssent
UDPPacketsreceived
UDPPacketsin LAN
Figure 27: UDP statistics in the course of a day ( Monday )
0
5
10
15
20
25
30
21-01
-08 00
:20
21-01
-08 01
:20
21-01
-08 02
:20
21-01
-08 03
:20
21-01
-08 04
:20
21-01
-08 05
:20
21-01
-08 06
:20
21-01
-08 07
:20
21-01
-08 08
:20
21-01
-08 09
:20
21-01
-08 10
:20
21-01
-08 11
:20
21-01
-08 12
:20
21-01
-08 13
:20
21-01
-08 14
:20
21-01
-08 15
:20
21-01
-08 16
:20
21-01
-08 17
:20
21-01
-08 18
:20
21-01
-08 19
:20
21-01
-08 20
:20
21-01
-08 21
:20
21-01
-08 22
:20
21-01
-08 23
:20
Time
ICM
P Pa
cket
s
Figure 28: ICMP packet count in the course of a day ( Monday )
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 53
0
5
10
15
20
25
30
21-01
-08 00
:00
21-01
-08 01
:10
21-01
-08 02
:20
21-01
-08 03
:30
21-01
-08 04
:40
21-01
-08 05
:50
21-01
-08 07
:00
21-01
-08 08
:10
21-01
-08 09
:20
21-01
-08 10
:30
21-01
-08 11
:40
21-01
-08 12
:50
21-01
-08 14
:00
21-01
-08 15
:10
21-01
-08 16
:20
21-01
-08 17
:30
21-01
-08 18
:40
21-01
-08 19
:50
21-01
-08 21
:00
21-01
-08 22
:10
21-01
-08 23
:20
Time
ICM
P Pa
cket
sTotalICMPPackets
ICMPPacketssent
ICMPPacketsreceived
ICMPPackets inLAN
Figure 29: ICMP packet count in the course of a day ( Monday )
0
50
100
150
200
250
300
350
21-01-0
8 00:00
21-01-0
8 00:50
21-01-0
8 01:40
21-01-0
8 02:30
21-01-0
8 03:20
21-01-0
8 04:10
21-01-0
8 05:00
21-01-0
8 05:50
21-01-0
8 06:40
21-01-0
8 07:30
21-01
-08 08:2
0
21-01
-08 09:1
0
21-01
-08 10:0
0
21-01
-08 10:5
0
21-01-0
8 11:4
0
21-01-0
8 12:30
21-01-0
8 13:20
21-01-0
8 14:10
21-01
-08 15:0
0
21-01-0
8 15:50
21-01-0
8 16:40
21-01-0
8 17:30
21-01-0
8 18:20
21-01-0
8 19:10
21-01-0
8 20:00
21-01-0
8 20:50
21-01
-08 21:4
0
21-01
-08 22:3
0
21-01
-08 23:2
0
Time
Con
nect
ions
cou
nt
Figure 30: Number of connections in the course of a day (Monday)
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 54
0
100
200
300
400
500
600
700
800
900
21-0
1-08
00:
00
21-0
1-08
06:
10
21-0
1-08
12:
20
21-0
1-08
18:
30
22-0
1-08
00:
40
22-0
1-08
06:
50
22-0
1-08
13:
00
22-0
1-08
19:
10
23-0
1-08
01:
20
23-0
1-08
07:
30
23-0
1-08
13:
40
23-0
1-08
19:
50
24-0
1-08
02:
00
24-0
1-08
08:
10
24-0
1-08
14:
20
24-0
1-08
20:
30
25-0
1-08
02:
40
25-0
1-08
08:
50
25-0
1-08
15:
00
25-0
1-08
20:
50
26-0
1-08
03:
00
26-0
1-08
09:
10
26-0
1-08
15:
20
26-0
1-08
21:
30
27-0
1-08
03:
40
27-0
1-08
09:
50
27-0
1-08
16:
00
27-0
1-08
22:
10
Time
Con
nect
ion
coun
t
Figure 31: Connection statistics in the course of a day (Monday)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
2-02-08
00:00
2-02-08
01:00
2-02-08
02:00
2-02-08
03:00
2-02-08
04:00
2-02-08
05:00
2-02-08
06:00
2-02-08
07:00
2-02-08
08:00
2-02-08
09:00
2-02-08
10:00
2-02-08
11:00
2-02-08
12:00
2-02-08
13:00
2-02-08
14:00
2-02-08
15:00
2-02-08
16:00
2-02-08
17:00
2-02-08
18:00
2-02-08
19:00
2-02-08
20:00
2-02-08
21:00
2-02-08
22:00
2-02-08
23:00
Time
Pack
et c
ount
Figure 32: Traffic statistics in the course of a day ( Saturday )
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 55
0
200
400
600
800
1000
1200
1400
1600
1800
2000
27-01
-08 00
:00
27-01
-08 01
:00
27-01
-08 02
:00
27-01
-08 03
:00
27-01
-08 04
:00
27-01
-08 05
:00
27-01
-08 06
:00
27-01
-08 07
:00
27-01
-08 08
:00
27-01
-08 09
:00
27-01
-08 10
:00
27-01
-08 11
:00
27-01
-08 12
:00
27-01
-08 13
:00
27-01
-08 14
:00
27-01
-08 15
:00
27-01
-08 16
:00
27-01
-08 17
:00
27-01
-08 18
:00
27-01
-08 19
:00
27-01
-08 20
:00
27-01
-08 21
:00
27-01
-08 22
:00
27-01
-08 23
:00
Time
Pack
et c
ount
Figure 33: Traffic statistics in the course of a day ( Sunday )
0
1000
2000
3000
4000
5000
6000
7000
21-0
1-08
00:
00
21-0
1-08
06:
00
21-0
1-08
12:
00
21-0
1-08
18:
00
22-0
1-08
00:
00
22-0
1-08
06:
00
22-0
1-08
12:
00
22-0
1-08
18:
00
23-0
1-08
00:
00
23-0
1-08
06:
00
23-0
1-08
12:
00
23-0
1-08
18:
00
24-0
1-08
00:
00
24-0
1-08
06:
00
24-0
1-08
12:
00
24-0
1-08
18:
00
25-0
1-08
00:
00
25-0
1-08
06:
00
25-0
1-08
12:
00
25-0
1-08
17:
40
25-0
1-08
23:
40
26-0
1-08
05:
40
26-0
1-08
11:
40
26-0
1-08
17:
40
26-0
1-08
23:
40
27-0
1-08
05:
40
27-0
1-08
11:
40
27-0
1-08
17:
40
27-0
1-08
23:
40
Time
Pack
et c
ount
Figure 34: Traffic statistics in the course of a week
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 56
0
1000
2000
3000
4000
5000
6000
7000
8000
19-0
1-08
10:
00
20-0
1-08
16:
20
21-0
1-08
22:
40
23-0
1-08
05:
00
24-0
1-08
11:
20
25-0
1-08
17:
20
26-0
1-08
23:
40
28-0
1-08
06:
00
29-0
1-08
12:
20
30-0
1-08
18:
50
1-0
2-08
02:
00
2-0
2-08
08:
20
3-0
2-08
14:
40
4-0
2-08
21:
00
6-0
2-08
03:
20
7-0
2-08
09:
40
8-0
2-08
16:
00
13-0
2-08
22:
40
15-0
2-08
05:
00
16-0
2-08
11:
30
19-0
2-08
17:
50
26-0
2-08
00:
20
27-0
2-08
06:
40
28-0
2-08
13:
00
2-0
3-08
19:
20
Time
Pack
et c
ount
Figure 35: Average Traffic statistics in the course of a month
0
20000
40000
60000
80000
100000
120000
17-03
-08 0
0:31
17-03
-08 0
1:41
17-03
-08 0
2:51
17-03
-08 0
4:01
17-03
-08 0
5:11
17-03
-08 0
6:22
17-03
-08 0
7:32
17-03
-08 0
8:42
17-03
-08 0
9:52
17-03
-08 1
1:02
17-03
-08 1
2:12
17-03
-08 1
3:22
17-03
-08 1
4:32
17-03
-08 1
5:42
17-03
-08 1
6:52
17-03
-08 1
8:02
17-03
-08 1
9:12
17-03
-08 2
0:22
17-03
-08 2
1:32
17-03
-08 2
2:42
17-03
-08 2
3:52
Time
Pack
et c
ount
Figure 36: Intrusive Traffic statistics in the course of a day (Monday)
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 57
0
20000
40000
60000
80000
100000
120000
140000
17-0
3-08
00:
01
17-0
3-08
07:
02
17-0
3-08
14:
02
17-0
3-08
21:
02
18-0
3-08
04:
03
18-0
3-08
11:
04
18-0
3-08
18:
05
19-0
3-08
01:
05
19-0
3-08
08:
05
19-0
3-08
15:
05
19-0
3-08
22:
06
20-0
3-08
05:
06
20-0
3-08
12:
07
20-0
3-08
19:
07
21-0
3-08
02:
07
21-0
3-08
09:
08
21-0
3-08
16:
08
21-0
3-08
23:
08
22-0
3-08
06:
08
22-0
3-08
13:
09
22-0
3-08
20:
10
23-0
3-08
03:
10
23-0
3-08
10:
10
23-0
3-08
17:
11
Time
Pack
et c
ount
Figure 37: Intrusive Traffic statistics in the course of a week
0
500
1000
1500
2000
2500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Time
Pack
et c
ount
AverageTCPPacketcount
AverageUDPPacketcount
AverageICMPPacketcount
Figure 38: Average Traffic statistics in the course of a day (Monday )
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 58
6.2. SCREENSHOTS OF BASE CONSOLE
Figure 39: BASE Console displaying the Traffic statistics by protocol
Figure 40: BASE console displaying the alerts statistics
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 59
Figure 41: BASE console displaying unique alerts
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 60
6.3. TYPICAL VALUESOF NETWORK PARAMETERS FOR NORMAL
TRAFFIC IN THE TARGET NETWORK
2 3 - 0 1 - 0 8 0 0 : 0 0 , W e d , 3 9 5 , 1 5 4 , 2 3 7 , 4 , 6 9 , 2 8 , 3 6 , 5 , 1 1 , 2 , 4 , 4 , 4 7 , 8 5 , 9 5 , 4 , 7 , 0 . 4 3 , 0 . 7 3 , 0 . 2 3 , 0 . 3 8 , 0 . 0 8 , 0 . 1 5 , 0 . 0 1 , 0 . 0 1
2 3 - 0 1 - 0 8 0 0 : 1 0 , W e d , 4 0 5 , 1 5 8 , 2 4 3 , 4 , 7 1 , 2 8 , 3 7 , 5 , 1 1 , 2 , 5 , 5 , 4 9 , 8 7 , 9 7 , 4 , 7 , 0 . 4 4 , 0 . 7 5 , 0 . 2 4 , 0 . 3 8 , 0 . 0 8 , 0 . 1 5 , 0 . 0 1 , 0 . 0 1
2 3 - 0 1 - 0 8 0 0 : 2 0 , W e d , 3 1 3 , 1 2 2 , 1 8 8 , 3 , 5 5 , 2 2 , 2 9 , 4 , 9 , 2 , 4 , 4 , 3 8 , 6 7 , 7 5 , 3 , 6 , 0 . 3 4 , 0 . 5 8 , 0 . 2 1 , 0 . 3 4 , 0 . 0 7 , 0 . 1 3 , 0 . 0 1 , 0 . 0 2
2 3 - 0 1 - 0 8 0 0 : 3 0 , W e d , 2 9 0 , 1 1 3 , 1 7 4 , 3 , 5 1 , 2 0 , 2 7 , 4 , 8 , 2 , 3 , 3 , 3 5 , 6 2 , 7 0 , 3 , 5 , 0 . 3 2 , 0 . 5 4 , 0 . 2 0 , 0 . 3 2 , 0 . 0 7 , 0 . 1 3 , 0 . 0 2 , 0 . 0 2
2 3 - 0 1 - 0 8 0 0 : 4 0 , W e d , 2 4 8 , 9 7 , 1 4 9 , 2 , 4 3 , 1 7 , 2 3 , 3 , 7 , 1 , 3 , 3 , 3 0 , 5 3 , 6 0 , 3 , 5 , 0 . 3 4 , 0 . 5 8 , 0 . 2 3 , 0 . 3 7 , 0 . 0 8 , 0 . 1 4 , 0 . 0 2 , 0 . 0 3
2 3 - 0 1 - 0 8 0 0 : 5 0 , W e d , 1 8 9 , 7 4 , 1 1 3 , 2 , 3 3 , 1 3 , 1 7 , 2 , 5 , 1 , 2 , 2 , 2 3 , 4 1 , 4 5 , 2 , 3 , 0 . 2 6 , 0 . 4 4 , 0 . 2 0 , 0 . 3 3 , 0 . 0 7 , 0 . 1 3 , 0 . 0 4 , 0 . 0 5
2 3 - 0 1 - 0 8 0 1 : 0 0 , W e d , 2 0 8 , 8 1 , 1 2 5 , 2 , 3 6 , 1 5 , 1 9 , 3 , 6 , 1 , 2 , 2 , 2 5 , 4 5 , 5 0 , 2 , 4 , 0 . 2 8 , 0 . 4 8 , 0 . 2 1 , 0 . 3 4 , 0 . 0 7 , 0 . 1 3 , 0 . 0 3 , 0 . 0 4
2 3 - 0 1 - 0 8 0 1 : 1 0 , W e d , 1 8 0 , 7 0 , 1 0 8 , 2 , 3 2 , 1 3 , 1 7 , 2 , 6 , 1 , 3 , 3 , 2 2 , 3 9 , 4 3 , 2 , 3 , 0 . 2 5 , 0 . 4 2 , 0 . 2 0 , 0 . 3 2 , 0 . 0 7 , 0 . 1 2 , 0 . 0 4 , 0 . 0 5
2 3 - 0 1 - 0 8 0 1 : 2 0 , W e d , 1 2 2 , 4 8 , 7 3 , 1 , 2 4 , 1 0 , 1 3 , 2 , 5 , 1 , 2 , 2 , 1 5 , 2 6 , 2 9 , 1 , 3 , 0 . 1 7 , 0 . 2 8 , 0 . 1 6 , 0 . 2 6 , 0 . 0 5 , 0 . 0 9 , 0 . 0 5 , 0 . 0 7
2 3 - 0 1 - 0 8 0 1 : 3 0 , W e d , 1 0 5 , 4 1 , 6 3 , 1 , 2 1 , 8 , 1 1 , 2 , 4 , 1 , 2 , 2 , 1 3 , 2 3 , 2 5 , 1 , 2 , 0 . 1 4 , 0 . 2 4 , 0 . 1 5 , 0 . 2 4 , 0 . 0 5 , 0 . 0 9 , 0 . 0 6 , 0 . 0 9
2 3 - 0 1 - 0 8 0 1 : 4 0 , W e d , 8 9 , 3 5 , 5 3 , 1 , 1 8 , 7 , 9 , 1 , 4 , 1 , 1 , 1 , 1 1 , 1 9 , 2 1 , 1 , 2 , 0 . 1 2 , 0 . 2 1 , 0 . 1 4 , 0 . 2 2 , 0 . 0 5 , 0 . 0 8 , 0 . 0 8 , 0 . 1 1
2 3 - 0 1 - 0 8 0 1 : 5 0 , W e d , 9 1 , 3 5 , 5 5 , 1 , 1 8 , 7 , 1 0 , 1 , 4 , 1 , 1 , 1 , 1 1 , 2 0 , 2 2 , 1 , 2 , 0 . 1 2 , 0 . 2 1 , 0 . 1 4 , 0 . 2 3 , 0 . 0 5 , 0 . 0 8 , 0 . 0 8 , 0 . 1 1
2 3 - 0 1 - 0 8 0 2 : 0 0 , W e d , 7 9 , 3 1 , 4 7 , 1 , 1 6 , 6 , 8 , 1 , 3 , 1 , 1 , 1 , 9 , 1 7 , 1 9 , 1 , 2 , 0 . 4 3 , 0 . 7 3 , 0 . 5 2 , 0 . 8 4 , 0 . 1 7 , 0 . 3 1 , 0 . 0 2 , 0 . 0 3
2 3 - 0 1 - 0 8 0 2 : 1 0 , W e d , 8 1 , 3 2 , 4 9 , 1 , 1 6 , 6 , 9 , 1 , 3 , 1 , 1 , 1 , 1 0 , 1 7 , 1 9 , 1 , 2 , 0 . 4 4 , 0 . 7 5 , 0 . 5 3 , 0 . 8 5 , 0 . 1 7 , 0 . 3 1 , 0 . 0 1 , 0 . 0 2
2 3 - 0 1 - 0 8 0 2 : 2 0 , W e d , 6 0 , 2 3 , 3 6 , 1 , 1 2 , 5 , 6 , 1 , 2 , 1 , 1 , 0 , 7 , 1 3 , 1 4 , 1 , 1 , 0 . 3 3 , 0 . 5 6 , 0 . 4 6 , 0 . 7 3 , 0 . 1 5 , 0 . 2 7 , 0 . 0 2 , 0 . 0 3
2 3 - 0 1 - 0 8 0 2 : 3 0 , W e d , 5 5 , 2 1 , 3 3 , 1 , 1 1 , 4 , 6 , 1 , 2 , 1 , 1 , 0 , 7 , 1 2 , 1 3 , 1 , 1 , 0 . 3 0 , 0 . 5 1 , 0 . 4 4 , 0 . 7 0 , 0 . 1 4 , 0 . 2 5 , 0 . 0 2 , 0 . 0 3
2 3 - 0 1 - 0 8 0 2 : 4 0 , W e d , 6 1 , 2 4 , 3 7 , 1 , 1 2 , 5 , 6 , 1 , 2 , 1 , 1 , 0 , 7 , 1 3 , 1 5 , 1 , 1 , 0 . 3 3 , 0 . 5 7 , 0 . 4 6 , 0 . 7 4 , 0 . 1 5 , 0 . 2 7 , 0 . 0 2 , 0 . 0 3
2 3 - 0 1 - 0 8 0 2 : 5 0 , W e d , 5 6 , 2 2 , 3 4 , 1 , 1 1 , 4 , 6 , 1 , 2 , 1 , 1 , 0 , 7 , 1 2 , 1 3 , 1 , 1 , 0 . 3 1 , 0 . 5 2 , 0 . 4 4 , 0 . 7 1 , 0 . 1 4 , 0 . 2 6 , 0 . 0 2 , 0 . 0 3
2 3 - 0 1 - 0 8 0 3 : 0 0 , W e d , 4 2 , 1 6 , 2 5 , 0 , 8 , 3 , 4 , 1 , 2 , 1 , 1 , 0 , 5 , 9 , 1 0 , 1 , 1 , 0 . 2 3 , 0 . 3 9 , 0 . 3 8 , 0 . 6 1 , 0 . 1 3 , 0 . 2 2 , 0 . 0 3 , 0 . 0 5
2 3 - 0 1 - 0 8 0 3 : 1 0 , W e d , 4 4 , 1 7 , 2 6 , 0 , 9 , 4 , 5 , 1 , 2 , 1 , 1 , 0 , 5 , 9 , 1 1 , 1 , 1 , 0 . 2 4 , 0 . 4 1 , 0 . 3 9 , 0 . 6 3 , 0 . 1 3 , 0 . 2 3 , 0 . 0 3 , 0 . 0 4
2 3 - 0 1 - 0 8 0 3 : 2 0 , W e d , 4 3 , 1 7 , 2 6 , 0 , 9 , 3 , 5 , 1 , 2 , 1 , 1 , 0 , 5 , 2 4 , 1 0 , 1 , 1 , 0 . 2 3 , 0 . 4 0 , 0 . 2 4 , 0 . 6 2 , 0 . 1 3 , 0 . 2 3 , 0 . 0 3 , 0 . 0 4
2 3 - 0 1 - 0 8 0 3 : 3 0 , W e d , 4 0 , 1 6 , 2 4 , 0 , 8 , 3 , 4 , 1 , 2 , 1 , 1 , 0 , 5 , 2 2 , 1 0 , 1 , 1 , 0 . 4 4 , 0 . 7 4 , 0 . 4 7 , 1 . 2 0 , 0 . 2 4 , 0 . 4 3 , 0 . 0 7 , 0 . 1 0
2 3 - 0 1 - 0 8 0 3 : 4 0 , W e d , 2 8 , 1 1 , 1 7 , 0 , 6 , 2 , 3 , 0 , 1 , 1 , 1 , 0 , 3 , 1 5 , 7 , 0 , 1 , 0 . 3 1 , 0 . 5 2 , 0 . 3 9 , 1 . 0 0 , 0 . 2 0 , 0 . 3 6 , 0 . 1 2 , 0 . 1 7
2 3 - 0 1 - 0 8 0 3 : 5 0 , W e d , 3 4 , 1 3 , 2 0 , 0 , 7 , 3 , 4 , 1 , 2 , 1 , 1 , 0 , 4 , 1 9 , 8 , 1 , 1 , 0 . 3 7 , 0 . 6 3 , 0 . 4 3 , 1 . 1 0 , 0 . 2 3 , 0 . 4 0 , 0 . 0 9 , 0 . 1 3
2 3 - 0 1 - 0 8 0 4 : 0 0 , W e d , 3 2 , 1 3 , 1 9 , 0 , 6 , 3 , 3 , 0 , 2 , 1 , 1 , 0 , 4 , 1 8 , 8 , 1 , 1 , 0 . 3 6 , 0 . 6 1 , 0 . 4 3 , 1 . 1 1 , 0 . 2 2 , 0 . 4 0 , 0 . 1 0 , 0 . 1 4
2 3 - 0 1 - 0 8 0 4 : 1 0 , W e d , 2 5 , 1 0 , 1 5 , 0 , 5 , 2 , 3 , 0 , 1 , 0 , 0 , 0 , 3 , 1 4 , 6 , 0 , 1 , 0 . 2 8 , 0 . 4 8 , 0 . 3 8 , 0 . 9 8 , 0 . 2 0 , 0 . 3 5 , 0 . 1 5 , 0 . 2 1
2 3 - 0 1 - 0 8 0 4 : 2 0 , W e d , 3 6 , 1 4 , 2 1 , 0 , 7 , 3 , 4 , 1 , 2 , 1 , 1 , 0 , 4 , 2 0 , 8 , 1 , 1 , 0 . 4 0 , 0 . 6 9 , 0 . 4 5 , 1 . 1 8 , 0 . 2 4 , 0 . 4 2 , 0 . 0 9 , 0 . 1 2
2 3 - 0 1 - 0 8 0 4 : 3 0 , W e d , 3 9 , 1 6 , 2 3 , 0 , 8 , 3 , 4 , 1 , 2 , 1 , 1 , 0 , 5 , 2 1 , 9 , 1 , 1 , 0 . 4 4 , 0 . 7 4 , 0 . 4 7 , 1 . 2 2 , 0 . 2 5 , 0 . 4 4 , 0 . 0 8 , 0 . 1 1
2 3 - 0 1 - 0 8 0 4 : 4 0 , W e d , 3 7 , 1 5 , 2 2 , 0 , 7 , 3 , 4 , 1 , 2 , 1 , 1 , 0 , 4 , 2 0 , 9 , 1 , 1 , 0 . 4 1 , 0 . 7 0 , 0 . 4 6 , 1 . 1 9 , 0 . 2 4 , 0 . 4 3 , 0 . 0 8 , 0 . 1 2
2 3 - 0 1 - 0 8 0 4 : 5 0 , W e d , 4 5 , 1 8 , 2 7 , 0 , 9 , 4 , 5 , 1 , 2 , 0 , 1 , 1 , 5 , 2 5 , 1 1 , 1 , 1 , 0 . 1 3 , 0 . 2 1 , 0 . 1 3 , 0 . 3 3 , 0 . 0 7 , 0 . 1 2 , 0 . 0 2 , 0 . 0 2
2 3 - 0 1 - 0 8 0 5 : 0 0 , W e d , 4 9 , 2 0 , 2 9 , 0 , 1 0 , 4 , 4 , 1 , 2 , 0 , 1 , 1 , 6 , 2 7 , 1 2 , 1 , 1 , 0 . 1 4 , 0 . 2 3 , 0 . 1 3 , 0 . 3 4 , 0 . 0 7 , 0 . 1 3 , 0 . 0 1 , 0 . 0 2
2 3 - 0 1 - 0 8 0 5 : 1 0 , W e d , 4 2 , 1 7 , 2 5 , 0 , 8 , 3 , 4 , 1 , 2 , 0 , 1 , 1 , 5 , 2 3 , 1 0 , 1 , 1 , 0 . 1 2 , 0 . 2 0 , 0 . 1 2 , 0 . 3 2 , 0 . 0 6 , 0 . 1 2 , 0 . 0 2 , 0 . 0 2
2 3 - 0 1 - 0 8 1 7 : 4 0 , W e d , 0 3 , 4 2 , 6 0 , 1 , 7 2 , 2 9 , 4 3 , 1 , 1 0 , 2 , 4 , 4 , 1 2 , 5 7 , 2 4 , 6 , 9 , 0 . 1 0 , 0 . 1 7 , 0 . 0 7 , 0 . 1 8 , 0 . 0 2 , 0 . 0 3 , 0 . 0 0 , 0 . 0 0
2 3 - 0 1 - 0 8 1 7 : 5 0 , W e d , 9 1 , 3 7 , 5 3 , 1 , 6 4 , 2 5 , 3 7 , 1 , 9 , 2 , 4 , 4 , 1 1 , 5 0 , 2 1 , 5 , 7 , 0 . 0 9 , 0 . 1 5 , 0 . 0 6 , 0 . 1 7 , 0 . 0 2 , 0 . 0 3 , 0 . 0 0 , 0 . 0 0
2 3 - 0 1 - 0 8 1 8 : 0 0 , W e d , 9 5 , 3 9 , 5 5 , 1 , 6 7 , 2 7 , 3 9 , 1 , 9 , 2 , 4 , 4 , 1 1 , 5 2 , 2 2 , 5 , 8 , 0 . 1 4 , 0 . 2 4 , 0 . 1 0 , 0 . 2 6 , 0 . 0 3 , 0 . 0 5 , 0 . 0 0 , 0 . 0 0
2 3 - 0 1 - 0 8 1 8 : 1 0 , W e d , 1 0 7 , 4 4 , 6 2 , 1 , 7 5 , 3 0 , 4 4 , 1 , 1 0 , 2 , 4 , 4 , 1 2 , 5 9 , 2 5 , 6 , 9 , 0 . 1 6 , 0 . 2 7 , 0 . 1 1 , 0 . 2 8 , 0 . 0 3 , 0 . 0 5 , 0 . 0 0 , 0 . 0 0
2 3 - 0 1 - 0 8 1 8 : 2 0 , W e d , 8 2 , 3 4 , 4 8 , 1 , 5 7 , 2 3 , 3 4 , 1 , 8 , 2 , 3 , 3 , 1 0 , 4 5 , 1 9 , 5 , 7 , 0 . 1 2 , 0 . 2 1 , 0 . 0 9 , 0 . 2 4 , 0 . 0 3 , 0 . 0 4 , 0 . 0 0 , 0 . 0 0
23 - 01 -0 8 18 :3 0 ,We d , 19 8 ,8 1 , 1 15 ,2 ,1 3 9 , 55 , 83 ,0 ,1 9 , 4 , 8 , 8 , 2 3 , 1 0 9 , 46 ,1 1 , 1 7 , 0 . 3 0 , 0 . 51 ,0 .1 4 ,0 .3 8 , 0 . 0 4 , 0 . 07 , 0 . 00 ,0 .0 0
2 3 - 0 1 - 0 8 1 8 : 4 0 , W e d , 2 1 1 , 8 7 , 1 2 2 , 2 , 5 3 , 2 1 , 3 0 , 1 , 7 , 1 , 3 , 3 , 2 4 , 1 2 7 , 4 9 , 4 , 6 , 0 . 3 2 , 0 . 5 4 , 0 . 1 4 , 0 . 3 9 , 0 . 0 7 , 0 . 1 2 , 0 . 0 1 , 0 . 0 1
24-01-08 09 :00,Th u,2164,844,1298,22 ,108 ,32 ,68,7 ,13 ,5,5 ,3,325,464,519,6 ,14 ,0.26 ,0 .44,0 .06 ,0.10 ,0 .05,0 .06 ,0.00,0 .00
24-01-08 09:10,Thu,5349,2086,3209,53,267,80,177,10,32,13,13,6,802,1043,1284,16,35,0.64,1.08,0.10,0.15,0.07,0.10,0.00,0.00
24-01-08 09:20,Thu,2896,1129 ,1738,29,145,43,90 ,12,17,7,7 ,3,434 ,565,695,9 ,18,0 .35,0 .59,0 .07,0 .11,0 .05,0 .07,0 .00,0 .00
24-01-08 09:30,Thu,3398,1325,2039,34,170,51,107,12,20,8,8,4,510 ,663,816,10,21,0.41,0.69,0.08,0.12,0.06,0.08,0.00,0.00
24-01-08 09:40,Thu,4100,1599 ,2460,41,103,31,59 ,13,12,5,5 ,2,615 ,800,984,6 ,12,0 .49,0 .83,0 .09,0 .13,0 .09,0 .13,0 .00,0 .01
24-01-08 09 :50 ,Th u,2954 ,1152 ,1772 ,30 ,74 ,22,39 ,12 ,9 ,4,4 ,2 ,354 ,576 ,709 ,4,8 ,0 .35 ,0.60 ,0 .07 ,0.11 ,0 .07 ,0.11 ,0 .01 ,0.01
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 61
6.4. TYPICAL VALUES OF NETWORK PARAMETERS FOR
ANOMALOUS TRAFFIC IN THE TARGET NETWORK
12/3/2008 17:11,Wed, 4028,1175,2451,402,290,131,107,52,30,11,13,6,346,148,192,35,30,0.59,0.89,0.39,0.5,0.17,0.25,0.08,0.01
12/3/2008 17:21,Wed, 1454,270,1040,144,271,122,100,49,16,6,7,3,135,46,44,33,27,0.57,0.85,0.39,0.49,0.17,0.25,0.08,0.01
12/3/2008 17:32,Wed, 3686,1370,1928,388,330,149,122,59,48,18,21,10,623,178,224,94,82,0.88,1.32,0.48,0.61,0.22,0.29,0.09,0.04
12/3/2008 17:42,Wed, 7322,2130,4422,770,466,210,172,84,86,32,37,17,1331,253,349,33,29,0.52,0.78,0.37,0.47,0.17,0.22,0.07,0.09
12/3/2008 17:52,Wed,601,7504,3851,2864,790,498,224,184,90,32,12,14,6,1925,442,630,17,14,0.36,0.54,0.31,0.39,0.14,0.19,0.06,0.15
12/3/2008 18:02,Wed, 9143,1857,6324,961,298,134,110,54,2,1,1,0,844,222,304,22,18,0.41,0.61,0.32,0.41,0.15,0.19,0.06,0.12
12/3/2008 18:12,Wed,601,9454,1212,7249,993,282,127,104,51,34,13,15,7,865,149,198,20,16,0.38,0.57,0.31,0.39,0.14,0.19,0.06,0.13
12/3/2008 18:22,Wed, 8045,614,6587,843,273,123,101,49,12,4,5,2,279,84,101,26,22,0.44,0.66,0.33,0.42,0.15,0.2,0.06,0.1
12/3/2008 18:32,Wed, 13065,1149,10543,1372,456,205,169,82,19,7,8,4,638,154,188,73,64,0.76,1.14,0.44,0.55,0.2,0.27,0.08,0.05
12/3/2008 18:42,Wed,601,13086,1669,10042,1376,487,219,180,88,39,14,17,8,758,206,273,42,36,0.57,0.85,0.38,0.48,0.17,0.23,0.07,0.07
12/3/2008 18:52,Wed, 10534,729,8700,1105,415,187,154,75,20,7,9,4,405,97,119,27,23,0.45,0.67,0.33,0.42,0.15,0.2,0.06,0.02
12/3/2008 19:02,Wed, 24285,2408,19323,2554,556,250,206,100,38,14,16,8,1338,284,367,23,19,0.4,0.6,0.35,0.44,0.14,0.19,0.06,0.03
12/3/2008 19:12,Wed, 17342,14452,1065,1825,340,153,126,61,11,4,5,2,6569,1635,2203,16,13,0.33,0.5,0.32,0.4,0.13,0.17,0.05,0.04
12/3/2008 19:22,Wed, 25755,7473,15572,2710,388,175,144,70,11,4,5,2,4152,850,1139,15,12,0.97,1.45,0.93,1.19,0.39,0.51,0.16,0.07
12/3/2008 19:32,Wed, 27803,2970,21909,2924,319,144,118,57,49,18,21,10,1350,345,453,17,14,1.03,1.55,0.96,1.22,0.4,0.53,0.17,0.26
12/3/2008 19:42,Wed, 28156,8169,17024,2963,476,214,176,86,45,17,19,9,4084,945,1245,107,94,2.68,4.01,1.55,1.97,0.65,0.86,0.27,0.06
12/3/2008 19:52,Wed, 22538,1735,18433,2370,447,201,165,80,8,3,3,2,723,207,265,23,19,1.2,1.79,1.03,1.31,0.43,0.57,0.18,0.1
12/3/2008 20:02,Wed,601,32062,9301,19387,3374,431,194,159,78,2,1,1,0,6643,1055,1418,16,13,0.99,1.48,0.94,1.2,0.39,0.52,0.16,0.14
12/3/2008 20:12,Wed, 38772,6810,27882,4080,353,159,131,64,10,4,4,2,3095,773,1038,10,8,0.79,1.18,0.84,1.06,0.35,0.46,0.14,0.19
12/3/2008 20:22,Wed, 42546,2651,35419,4475,20379,9171,7540,3668,10,4,4,2,1326,305,404,8,6,0.67,1,0.78,0.99,0.32,0.43,0.13,0.25
12/3/2008 20:32,Wed, 29981,4164,22662,3155,397,179,147,71,13,5,6,3,2974,477,635,13,10,0.89,1.34,0.9,1.14,0.37,0.49,0.15,0.21
12/3/2008 23:32,Wed, 28394,23662,1743,2989,364,164,135,66,12,4,5,2,1816,2671,3608,352,313,0.78,1.17,0.64,0.82,0.16,0.18,0.06,0.02
12/3/2008 23:42,Wed, 39968,1461,34304,4203,492,221,182,89,8,3,3,2,150,174,223,198,176,0.58,0.87,0.56,0.71,0.14,0.16,0.05,0.03
12/3/2008 23:52,Wed,601,7845,329,6695,822,155,70,57,28,31,11,13,6,40,45,50,140,124,0.98,1.47,1.02,1.3,0.25,0.29,0.09,0.09
13-03-08 00:02,Thu, 392,327,65,0,23,10,9,4,0,0,0,0,48,43,50,97,85,0.81,1.22,0.93,1.18,0.23,0.27,0.08,0.07
13-03-08 00:12,Thu, 0,0,0,0,67,30,25,12,0,0,0,0,0,8,0,133,117,0.95,1.43,1.01,1.28,0.25,0.29,0.09,0.05
13-03-08 00:22,Thu, 9,6,3,0,91,41,34,16,1,0,0,0,0,10,1,226,201,1.19,1.77,1.12,1.43,0.28,0.31,0.1,0.04
13-03-08 00:32,Thu,601,5,4,2,0,91,41,34,16,0,0,0,0,0,10,1,241,214,1.22,1.83,1.14,1.44,0.28,0.32,0.1,0.04
13-03-08 00:42,Thu, 0,0,0,0,43,19,16,8,0,0,0,0,0,9,0,214,190,1.15,1.72,1.11,1.41,0.27,0.31,0.1,0.01
13-03-08 00:52,Thu,601,0,0,0,0,56,25,21,10,0,0,0,0,0,7,0,142,125,0.94,1.4,0.99,1.26,0.25,0.28,0.09,0.01
13-03-08 01:02,Thu,601,0,0,0,0,101,46,38,18,0,0,0,0,0,6,0,93,81,0.75,1.13,0.89,1.14,0.22,0.25,0.08,0.01
13-03-08 01:12,Thu, 7,1,6,0,86,39,32,15,0,0,0,0,0,7,0,121,107,0.87,1.3,0.96,1.22,0.24,0.27,0.08,0.01
13-03-08 01:22,Thu, 34,1,33,0,30,13,11,5,2,1,1,0,0,10,0,227,202,1.17,1.76,1.11,1.41,0.27,0.31,0.1,0.01
13-03-08 01:32,Thu, 14,10,4,0,70,32,26,13,0,0,0,0,1,9,2,164,145,1,1.49,1.02,1.3,0.25,0.28,0.09,0.01
13-03-08 01:42,Thu,601,11,9,2,0,80,36,30,14,0,0,0,0,1,9,1,175,156,1.03,1.54,1.04,1.32,0.26,0.29,0.09,0.01
13-03-08 01:52,Thu,601,0,0,0,0,46,21,17,8,0,0,0,0,0,7,0,117,103,0.42,0.63,0.4,0.51,0.12,0.13,0.04,0.01
13-03-08 02:02,Thu, 0,0,0,0,69,31,26,12,0,0,0,0,0,9,0,215,191,0.57,0.85,0.47,0.59,0.13,0.15,0.05,0
13-03-08 02:12,Thu, 0,0,0,0,57,26,21,10,0,0,0,0,0,14,0,61,53,0.9,1.35,0.59,0.75,0.29,0.33,0.1,0.02
13-03-08 02:22,Thu, 8,6,2,0,59,27,22,11,0,0,0,0,0,22,1,73,64,0.99,1.48,0.61,0.77,0.31,0.35,0.11,0.02
13-03-08 02:33,Thu,601,0,0,0,0,60,27,22,11,0,0,0,0,0,39,0,7,5,1.76,2.64,0.81,1.03,1.06,1.2,0.37,1.04
13-03-08 02:43,Thu, 0,0,0,0,48,22,18,9,0,0,0,0,0,61,0,14,11,2.75,4.12,1.02,1.29,1.32,1.5,0.47,0.53
13-03-08 02:53,Thu,601,39,24,15,0,39,18,14,7,2,1,1,0,1,32,4,5,3,1.32,1.98,0.68,0.87,0.9,1.02,0.32,1.47
13-03-08 03:03,Thu,602,43,20,23,0,80,36,30,14,0,0,0,0,1,19,3,49,43,0.78,1.17,0.53,0.67,0.27,0.3,0.09,0.03
13-03-08 03:13,Thu,601,9,8,1,0,51,23,19,9,4,1,2,1,1,11,1,17,14,0.89,1.33,0.93,1.19,0.4,0.46,0.14,0.13
13-03-08 03:23,Thu, 0,0,0,0,100,45,37,18,0,0,0,0,0,5,0,21,17,0.5,0.74,0.7,0.89,0.21,0.24,0.08,0.06
13-03-08 03:33,Thu, 109,100,9,0,47,21,18,9,2,1,1,0,12,17,15,19,16,0.48,0.72,0.69,0.88,0.21,0.24,0.07,0.06
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 62
7. APPENDIX B
7.1. GLOSSARY OF TECHNICAL TERMS
Alert
A message generated by IDS whenever it detects an event of
interest. An alert typically contains information about the attack
or some unusual activity that was detected
Anomaly Any significant deviations from the normal behaviour/pattern
Attack An intelligent act that is a deliberate attempt (especially in the
sense of a method or technique) to evade security services and
violate the security policy of a system In other words, an
intrusion attempt
Event
Activity detected by the IDS which may result in an alert. For
example, ‘N’ failed logins in ‘T’ seconds might indicate a brute-
force login attack
False negative
occurs if the IDS does not identify an event that is part of an
attack as being malicious
False positive
occurs if the IDS identifies an event that is not part of an attack
as being malicious
Intrusion Any set of actions that attempt to compromise the
confidentiality, integrity or availability of system or network
resources. Any intrusion is a consequence of an attack, but not
all attacks lead to an intrusion
Intrusion
Detection
System
Monitors computer systems and/or network and analyzes the
data for possible hostile attacks originating from external world
and also for system misuse or attacks originating from inside
the enterprise
Network
Security
Protection of Integrity, Availability and Confidentiality of
Network Assets and services from associated threats and
vulnerabilities so as to maintain the service availability, avoid
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 63
financial losses, damage to image, protect personnel, customer
and business secrets etc
Normalizing
Removal of unwanted strings from the data to reconstruct the
application layer payload
E.g. Telnet sessions contains telnet negotiation strings like IAC
(Interpret as Command), NOP (No Operation) etc, which can
disrupt the signature matching at Detection Engine. These
strings need to be normalized before passing them on to
Detection Module
Plug-in
A plug-in is a piece of code (written to comply with a particular
API) which extends the capability of a existing program or tool
like snort. Plug-in provide the ability to make snort do new and
interesting things without directly modifying the internal
architecture.
SNORT has three kinds of plug-ins. They are pre-processor
plug-in, detection plug-ins, and output plug-ins. Each of these
acts at a different point in the detection scheme. The pre-
processor plug-ins work on packets before they are passed to the
detection engine. The detection plug-ins are employed as part of
the rules used to match packets. The output plug-ins work with
either the alert messages or the packets to be logged
Promiscuous
Mode
Network Interface card when set in promiscuous mode, not only
accepts the packets intended to it but also receives and processes
all other packets which are moving around in the network
Sensor
Sensor is a part of the network Intrusion Detection that collects
data about activities from data sources, detects events, and
forwards them to the analyzer
Session
A session is a series of interactions between two communication
end points that occur during the span of a single connection.
Typically, one end point requests a connection with another
specified end point and if that end point replies agreeing to the
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 64
connection, the end points take turns exchanging commands
and data (talking to each other). The session begins when
the connection is established at both ends and terminates when
the connection is ended
Signature /
Pattern based
intrusion
detection
The intrusion detection system contains a database of known
vulnerabilities in the form a sequence of strings. It monitors
traffic and seeks a pattern or a signature match
SPAN
(Switched Port
Analyzer)
SPAN copies incoming and outgoing packets from multiple
sources, VLANs or ports, to a single destination port
Spoofing
A technique used to gain unauthorized access to computers,
whereby the intruder sends messages to a computer with an IP
address indicating that the message is coming from a trusted
host. To engage in IP spoofing, an attacker must first use a
variety of techniques to find an IP address of a trusted host and
then modify the packet header so that the packets appear to be
coming from the trusted host
True Negative
They occur when no alerts are triggered for events which are not
part of an attack(s)
True Positive They occur when alerts are triggered for events which are part
of an attack(s)
Vulnerability
A flaw or weakness in a system’s design, implementation, or
operation and management that could be exploited to violate the
system’s security posture
Security Policy
A set of rules and practices that specify or regulate how a system
or organization provides security services to protect sensitive
and critical system resources
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 65
8. APPENDIX C
8.1. ATTACK DESCRIPTION
Apache2 This attack exploits the inability of some versions of the Apache
web server to handle very long HTTP requests. A typical attack
contains multiple requests each with thousands of lines and
looking something like this:
GET / HTTP/1.1
User-Agent: sioux
User-Agent: sioux
ARPpoison
An attacker who has compromised a host on the local network
disrupts traffic by listening for “ARP-who-has” packets and
sending forged replies. ARP (address resolution protocol) is used
to resolve IP addresses to Ethernet addresses. Thus, the attacker
disrupts traffic by misdirecting traffic at the data link layer
DoS attack
A denial-of-service attack or distributed denial-of-service attack (DDoS
attack) is an attempt to make a computer resource unavailable to
its intended users. Although the means to, motives for, and
targets of a DoS attack may vary, it generally consists of the
concerted, malevolent efforts of a person or persons to prevent an
Internet site or service from functioning efficiently or at all,
temporarily or indefinitely by choking the network bandwidth,
and/or consuming computing resources like memory and CPU.
Fragment
overlap attack
A TCP/IP Fragmentation Attack is possible because IP allows
packets to be broken down into fragments for more efficient
transport across various media. The TCP packets (and its header)
are carried in the IP packet. In this attack the second fragment
contains incorrect offset. When packet is reconstructed, the port
number will be overwritten
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 66
IPsweep
An IPsweep attack is a surveillance sweep to determine which
hosts are listening on a network. This information is useful to an
attacker in staging attacks and searching for vulnerable machines
Land
This is a Denial of service attack where a remote host is sent a
UDP packet with the same source and destination
Mailbomb
This attack floods a user with thousands of junk emails. This
type of attack can be detected by the fact that the SMTP “mail”
command is lowercase. It is normally uppercase but not required
to be
Neptune Floods the target machine with SYN requests on one or more
ports, thus causing Denial of service
Phf attack
The Phf attack abuses a badly written CGI script to execute
commands with the privilege level of the http server. Any CGI
program which relies on the CGI function escape_shell_cmd() to
prevent exploitation of shell-based library calls may be
vulnerable to attack. In particular, this vulnerability is manifested
by the "phf" program that is distributed with the example code
for the Apache web server
PoD
This attack, also known as “ping of death”, crashes some older
operating system by sending an oversize fragmented IP packet
that reassembles to more than 65,535 bytes, the maximum
allowed by the IP protocol. It is called “ping of death” because
some older versions of Windows 95 could be used to launch the
attack using “ping -l 65510”
Smurf
This is a distributed network flooding attack initiated by sending
ICMP ECHO REQUEST packets to a broadcast address with the
spoofed source address of the target. The target is then flooded
with ECHO REPLY packets from every host on the broadcast
address
TCPreset
This attack listens for TCP SYN packets on a compromised host
on the local network and immediately sends a spoofed RST
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 67
(connection refused) packet, disrupting traffic
Teardrop
This attack reboots the Linux host by sending a fragmented IP
packet that cannot be reassembled because of a gap between the
fragments
UDPstorm
An attacker floods the local network by setting up a loop between
an echo server and a Client machine or another echo server by
sending a UDP packet to one server with the spoofed source
address of the other
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 68
9. APPENDIX D
9.1. THE TCP/IP PROTOCOL STACK
Source : http://www.tcpipguide.com/free/t_DataLinkLayerTechnologiesandProtocols.htm
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 69
9.2. IP HEADER
Source: http://www.visi.com/~mjb/Drawings/
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 70
9.3. TCP HEADER
Source: http://www.visi.com/~mjb/Drawings/
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 71
9.4. UDP HEADER
9.5. ICMP HEADER
Source: http://www.visi.com/~mjb/Drawings/
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 72
9.6. TCP CONNECTION ESTABLISHMENT
Source: http://www.tcpipguide.com/free/t_DataLinkLayerTechnologiesandProtocols.htm
Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur 73
9.7. TCP CONNECTION TERMINATION
Source : http://www.tcpipguide.com/free/t_DataLinkLayerTechnologiesandProtocols.htm