The Principles and Good Practices for Intrusion Prevention ... · The Principles and Good Practices...

© 2006 CLICO

The Principles and Good Practices for

Intrusion Prevention systems Design

by Mariusz Stawowski, CISSP email: [email protected]

12.11.2006.

The Principles and Good Practices for Intrusion Prevention systems Design

© 2006 CLICO 2

Table of Contents

1. INTRODUCTION 3

2. TYPICAL BREAK-IN SCENARIOS 5

3. THE PRINCIPLES AND STAGES OF INTRUSION PREVENTI ON SYSTEM DESIGN 9

4. DESIGNING THE SECURITY MANAGEMENT INFRASTRUCTUR E 15

5. ELABORATING IPS SECURITY POLICY 16

6. DESIGNING SCALABLE AND RELIABLE IPS PROTECTIONS 19

7. SECURITY INCIDENTS HANDLING 23

8. DESIGNING THE ACCEPTANCE TEST PLAN 24

Bibliography 25


© 2006 CLICO 3

1. Introduction

Ensuring a proper protection against threats from the network is the basic security requirement in IT systems nowadays. More and more advanced penetration and break-in techniques are being developed such as hybrid attacks as well as smart worms and trojans quickly spreading in the networks. Easy in use tools for the vulnerability exploitation are available in Internet. Old-generation security means such as network Intrusion Detection System (IDS) which rely on listening to the network or Stateful Inspection firewalls can not handle the task of protection against these threats.

`

Intruder performed an attack against IT system

Network IDS detected the attack

Intruder broken-in the IT system and installed the trojan

Network IDS signaled firewall about the attack from particular IP address

Firewall blocked IP address that launched the attack

Intruder gained access to IT system resources (e.g. through backdoor using another IP address)

Firewall

Network IDS

Internet/WAN

1

6

5

3

2

4

Intruder performed an attack against IT system

Network IPS detected the attack and blocked the packet containing malicious code Network IPS

Internet/WAN

1

2

`

Figure 1. Comparison of Network IPS system and the IDS integrated with a firewall


© 2006 CLICO 4

Security systems are required to detect attacks and to block them effectively. Security means of this class are callled IPS — Intrusion Prevention System. Their first implementation were Host IPS. However they have not been accepted for common use because they caused instabilities of servers they protected and in practice did not block many attacks. Since Host IPS software was installed directly on the servers, it created additional load and interfered in their work. An attack against computer system (e.g with exploit) is usually performed in a very short time. This is usually one-time connection to the server and execution of some malicious code on it. In practice, Network IDS security means, even those integrated with firewalls (using firewall signaling), can not block such attacks. The only protection means which are effective in this area are Network IPS. Although Network IDS integrated with firewall can be considered similar to the Network IPS, in reality it causes more threats than benefits. The difference can be best understood by analyzing common break-in to IT systems scenarios. The principles of working the two protection solutions is shown in figure 1. The comparison of effectiveness of the dedicated Network IPS and IDS integrated with firewall can be easily performed, even in a simple lab network conditions.

IPS systems working in an in-line mode can effectively and actively defy attacks of intruders. The principle of Network IPS work is similar to firewall systems. Traffic is coming into IPS device through a network interface, is analyzed in the device, and is coming out of the device using a separate network interface. Thanks to this, network traffic inspection is easier, there is less false-positive alerts than when using IDS, and most importantly, the threat that the IPS „losts packets” is completely eliminated. But the most important thing is that in the in-line mode, the attacks can be blocked in a real-time, even before they can reach the protected resources of the IT system.

The practical material about network intrusion prevention systems presented in this article are based on Juniper Networks Intrusion Detection and Prevention (IDP) solution. The traffic inspection process in IDP is shown in figure 2. The network traffic analysis in IDP use different detection techniques such as Stateful Signatures, Protocol Anomalies, Network Honeypot, Backdoor Detection, Traffic Anomalies, Spoofing Detection, Layer 2 Detection and Denial of Service Detection, used depending on the needs and configuration.

Data completing(i.e. assembling of

fragmented packets, retransmission reduction)

Data normalization(i.e. translation of

different formats and encoding systems)

Tracking of sessions and flows

(i.e. client-server, server-client, control

and data channels, TCP handshaking)

Detection and blocking of attacks and other

unauthorized activities

networkinterface

networkinterface

Figure 2. Process of network traffic inspection in the IDP security system


© 2006 CLICO 5

The basic security functionalities performed by the IPS security systems are the following:

− detecting and blocking penetrations and attacks conducted by intruders and Internet worms (i.e. automatic applications breaking into computer systems),

− monitoring of security status (such as detection of disallowed activities and notifying administrators about them),

− helping administrators in explaining security breaches (events correlation and generation of reports).

Advanced IPS solutions allow for deploying other security means. For instance, an IDP system is equipped, among others, with the following security mechanisms:

− cheating intruders using such techniques as ‘honeypot',

− blocking of spyware, keylogger and other malware applications,

− detecting of unauthorized computers connected to the internal networks,

− supervision of following by employees the accepted security policy rules (e.g checking if they use banned software),

− helping administrators in detecting systems and application vulnerable to the security bugs,

− detection of security incidents (e.g. identyfying backdoor connections using heuristic analysis techniques as well as newly opened ports on servers, specific for trojans).

2. Typical Break-in Scenarios

Deployment of an effective Intrusion Prevention system in an IT system requires knowledge of possible break-in scenarios and taking them into account in the security design. The IT system resources are vulnerable to threats of many types. Most often they result from a global character of it as well as from high level of complexity of computer network environments (e.g. Internet, intranet, extranet). Intruder attacks observed in the Internet usually start from reconnaissance and taking control over company’s public server (e.g. Web, SMTP, DNS). Such servers in most enterprises are located in a separated network segment (so called DMZ).

Breaking into the server is usually possible when there is some bug in its software allowing for execution of system commands. In this way a trojan application can be copied onto the server (e.g. CAFEiNi, G@du-Ghost, Luzak, SKUN, Wspomagacz) and executed in order to gain remote acess to the system. A bug in the server’s software is usually implemented in the application of special type commonly called as exploit. After taking control over the server, the intruder can perform various actions on it (e.g. Web graffiti) or continue attack against other security zones (e.g. internal network). A tactic of taking control over successive security zones in the network is called Island Hopping Attack.


© 2006 CLICO 6

Two typical break-in scenarios can be distinguished:

− direct break into the system using a bug in the network application (so called break-in through an exploit),

− breaking in using application sent into the internal network (so called break-in through an agent).

In case an attacker takes control over the system, he installs an intruder application in it and uses it to continue penetration of the internal networks. For this purpose a common trojan applications can be used or difficult to detect rootkit applications which work by replacing computer system files.

Figure 3. The concept of breaking into the system through an exploit

The most commonly used method in attacks through exploits is the buffer overflow. This technique works by writing a large amount of data to the application’s buffer in such a way that the buffer is overflowed, and the memory area after the buffer is overwritten. Usually in this way the application receives a new code to execute. The exploit attack works by putting malicious code into memory and executing it. Because of the type of buffers used in such attacks, the two attack techniques can be distinguished: stack smashing and heap smashing. There are also other known exploit techniques like format string overflow, however detailed description of this subject is bayond the scope of this article. Important for IPS designing is proper understanding of hacking technique concepts.

Internet/WAN

LAN

`

`

`

Server accessible from external network (e.g.

Web server)

Intruder

1. Intruder performed the attack (exploit) against network server

and installed the trojan

2. Intruder continues the attacks against

internal network systems


© 2006 CLICO 7

buffer

Returnaddress

Memory allocated to an application

Application controls the size of data input into the buffer

buffer

Returnaddress

Memory allocated to an application

Input data exceeded the buffer size

Malicious code(trojan)

Correct execution of an application Exploit attack

Figure 4. The concept of exploit type attack using buffer overflow

An application running in a computer has at its disposal a memory block of specific size which it uses for performing its functions. Application data is stored in memory areas called buffers. There is also an address stored in memory, which describes the location, the application goes after it finishes the current operation. This is so called return address. A properly written application controls data entered into the buffers and does not allow to exceed its allocated size. But if the application is not written properly, it allows for input data of size exceeding allowed buffer size. The concept of exploit type attack using buffer overflow has been presented in figure 4. An intruder writes into memory code for execution (e.g. running a backdoor on a specific TCP port) and modifies the return address of the function in a way that the code is executed.

A complete scenario of breaking into company’s internal network through an exploit can be performed in the following way:

− recognition of the attack target (e.g. port scanning, TCP/IP fingerprint),

− taking control over the system (usually a server located in the DMZ),

− installation of trojan/backdoor application,

− system penetration, data stealing and modification, and continuing the attack against other computer system resources.

When designing IPS network security system one should understand the circumstances of taking control over the system being attacked. After performing an attack with the exploit, loading and executing the shellcode, the intruder establishes connection with the system being attacked. It can be done by connection with a specific TCP/UDP port in which the shellcode has been executed or by the loopback connection to the intruder’s computer. The IPS security systems being designed should be prepared for such situations.


© 2006 CLICO 8

Breaking-into through an agent (or bot) works by sending malicious code to the company’s internal network. Usually this is done using generally available network services. For example JavaScript or ActiveX code can be included in HTML message and sent to the user by email. If the email client or Web browser used by the user is not up-to-date or is not properly configured (e.g. execution of ActiveX application is allowed without limitations), the code executed can copy user files or perform other dangerous activities. The code executed by the email client on the user’s computer can attempt to perform disallowed operations or start a web browser which will connect to the intruder’s computer and load, and execute malicious code (e.g. trojan application).

Internet/WAN LAN

`

`

`

Server with hidden Trojan

(e.g. Web server)

Intruder

Proxy server(e.g. Web, SMTP, P2P)

1. Unaware user downloaded the Trojan code

2. Trojan executed on user computer (e.g. exploiting Web browser bug)

3. Trojan connected to Proxy server

4. Intruder takes over control on Trojan and user computer

in internal network

Figure 5. The concept of breaking into through an agent

The agent’s code can also be sent to the internal network using more sophisticated methods such as Phishing or Pharming. Use of Phishing in this case works by sending to company’s employees email message on behalf of a trusted person (using Email Spoofing and Social Engineering), encouraging them to visit a specific Web page. For instance, an intruder presenting as an administrator or chief of an IT department sends an e-mail encouraging employees to visit a Web page in order to test a new application. In reality in the page there is exploit code against a Web browser with a trojan application. A Pharming technique works by attacking DNS service (DNS servers or local settings in the ‘hosts’ file stored in the computer) in such a way that in response to the DNS query for a trusted Web server, the IP address of the server prepared by the intruder is returned. This address points to the computer where the intruder stored an exploit code. This code attacks the user Web browser, and when it is executed, installs the agent in the user’s computer.


© 2006 CLICO 9

3. The Principles and Stages of Intrusion Preventi on system Design

Deploying of an effective, scallable and cost-effective IPS security solution for medium up to large scale IT system, requires its proper designing. Network IPS is an element of network security system which acts as an improvement and a complement of other security means used in the computer systems (e.g. firewall systems, protections in the operating systems, databases and applications). Before security solution deployment, the specification of its requirements should be performed (including the risk analysis). This should answer, among others, the following questions:

− which IT system resources should be protected against attacks (such as servers in the internal network, DMZ servers),

− what are the relevant resources threats and where are their sources (such as break-ins and DoS attacks from Internet, breaks-in from departamental networks through VPN),

− what should be an expected reaction to the attacks (e.g. alarming, attack blocking),

− where the protection means should be located (e.g. in the DMZ, before the firewall, in the internal network).

When designing the network security system, the following fundamental IT systems security principles1 should be taken into account [1-2, 4, 7, 9-11]:

− "Compartmentalization of Information" - computer system resources of different sensitivity levels (i.e. different value and threat susceptibility) should be located in different security zones. The extension to this rule is an "Information Hiding" principle, which says that an IT system makes available only such data which is necessary for conducting its tasks.

− "Defense-in-Depth" – protection of valuable IT system resources is based on many security layers. Extensions to this rule are the following rules2: "Layered Protections" – security layers complement and protect one another; "Defense in Multiple Places" – security layers shoud be placed in different IT system areas.

− "The Principle of Least Privilege" – users and administrators of IT system should have minimal privileges necessary for proper functioning of the institution. This rule applies also to data and services made available for external users. Extension to this rule is a "Need-To-Know" principle which says thar users and administrators of IT system have access to information relevant to their position and duties performed.

− "Weakest link in the chain" – a security level of the IT system depends on the most weakly protected element of the system.

1 When designing a network security system, the other rules applying to security organization should also be taken

into account. These are among others "Separation of Duty" and "Job Rotation" rules. Their task is to limit employees possibilities to break security rules. According to the "Separation of Duty" rule important tasks/functions should be performed by two or more employees. The "Job Rotation" principle says that there should be rotation of employees on important positions.

2 There is also known "Defense Through Diversification" principle, extending „Defense-in-Depth” rule. The principle states that the assets should be protected through multiple layers of security mechanisms, and should not rely on the components from single vendor in these different layers. In practice "Defense Through Diversification" principle should be used thoughtfully, as it makes the security management and maintenance more difficult.


© 2006 CLICO 10

Important for the intrusion prevention are properly designed network security zones and access control rules. User computers in internal networks should not have ability to directly access Internet, as Trojan sent to user computer should not have easy possibility to connect to intruder in external network (look at break-in scenario through an agent).

Internet

Web Cache

(HTTP Proxy)

Firewall

Mail

server

TROJAN

User computers

IPS, AV, ...SMTP, IMAP/POP3

HTTP, HTTPS

HTTP, HTTPS

SMTP

Figure 6. Internet access control of users in internal networks

The detailed rules of network security designing are derivied from „Compartmentalization of Information” principle:

− the devices and computer systems providing services to external networks (e.g. in Internet) should be located in different security zones as devices and computer systems of internal network (i.e. in DeMiliterized Zones - DMZ),

− strategic IT system resources should be located in the dedicated security zones,

− the devices and computer systems with low trust level like remote access devices (RAS) and wireless access points should be located in the dedicated security zones,

− the devices and computer systems providing services to customers and partners should be located in the dedicated security zones (i.e. Extranet zones),

− user workstations should be located in different security zones as servers,

− network and security management systems should be located in the dedicated security zones,

− systems and devices in the development should be located in different security zone as systems and devices in the production.


© 2006 CLICO 11

The stages of creating design of an Intrusion Prevention system has been presented below.

2. DETERMINING LOCATION OF IT SYSTEM’S RESOURCES REQUIRING IPS PROTECTION

3. DETERMINING THREATS TO THE RESOURCES AND SOURCES (LOCATION) OF THESE THREATS

4. DETERMINING LOCATION OF IPS PROTECTIONS

8. IPS MANAGEMENT INFRASTRUCTURE DESIGNING

9. ELABORATION OF SECURITY MANAGEMENT AND INCIDENT RESPONSE PROCEDURES

10. PROJECT ACCEPTANCE TESTS PLAN SPECIFICATION

1. DEFINITION OF PROJECT GOAL AND SCOPE

6. SELECTION OF IPS PRODUCTS FOR PROJECT IMPLEMENTATION

5. IPS SECURITY POLICY RULES FORMULATION

7. IPS SECURITY ARCHITECTURE DESIGNING

The crucial role of security means effectiveness plays their location. The IPS security solutions can be effective protections against network attacks when they are located between protected IT system resources and the sources of the threats (intruders). Security designing in the first place focuses on the most valuable IT system resources (i.e. computer systems performing or supporting business tasks of the organization). However protection should not be limited to the most valuable resources. The protections being designed should become an effective barrier against attacks conducted using the Island Hopping Attack technique. This technique works by gaining unathorized access to weaker protected areas (usually not very important for the organization) and using them as a base for penetration of better protected, more valuable IT system elements.


© 2006 CLICO 12

An identification of important threats to the IT system is performed, among others, based on the analysis of the following aspects:

− the way of connecting the system to the external networks,

− protocols and external network services available,

− protocols and services performed for the external networks,

− methods of co-operations between systems located in the network areas of different trust levels (e.g. DMZ and internal network).

When designing IPS system the threat of conducting attacks through encrypted sessions (e.g. HTTPS sessions in e-commerce and e-banking systems) should also be taken into account. Network protection can not control these sessions. An effective protection method is to terminate the SSL sessions and to inspect unencrypted packets. Deployinng IPS security for encrypted sessions require using devices called SSL Accelerators. From the technical point of view, HTTPS sessions can be decrypted on the IPS device after copying onto them private SSL keys, but on account of cryptographic material security it is reccommended that this operation is performed on the specialized devices.

Figure 6. The concept of e-commerce system protection against attacks through SSL

When determining security requirements for IT system resources one should take into consideration if they are mission critical resources, where the priority is continuity of operation or they are data-sensitive resources where data confidentiality is the most important factor. For mission critical resources the IPS protections should be designed in high availbility configurations.

Internet / WAN

E-commerceservers

Firewall

SSL AcceleratorsIPS

HTTPS HTTP


© 2006 CLICO 13

In the following example, the key stages of IPS protection design process have been presented (figures 7 and 8):

− determining IT system’s resources requiring IPS protection,

− determining threats to those resources and sources of these threats,

− deciding about location of IPS protection.

WAN

E-commerceserver

INTERNET

HQ domaincontroller

Applicationserver

Filesserver

Web and DNS intranet servers

Intranet mailserver

Databaseserver

Mobile usersDial-up / (W)LAN

Dial-up

HQ usersworkstations

RASserver

Internet mailserver

Web and DNS Internet servers

Authenticationserver

Internet Rou ter

Office networks

Production serversIntranet servers

Network security supporting systems

Internet servers

Personal Firewall

Perimeter Firewall Internal Firewall

WANrouter

Personal Firewall

- IT system resources requering IPS protection

- threat sources

- potential attacks paths

S2C

CertificateAuthority

Figure 7. IPS protection designing sample (determining resources which should be protected and identification of the threats and their sources)


© 2006 CLICO 14

WAN

E-commerceservers

INTERNET

Mobileusers

Dial-up/ (W)LAN

Dial-up


Internetrouter

Officenetworks

Personal Firewall

Perimeter Firewall Internal Firewall

WANrouter

Personal Firewall

SSL Accelerator

IPS

IPS (2 VS) IPS (2 VS)

HQ domaincontroller

Applicationserver

Filesserver


Intranet mailserver

Databaseserver

RASserver

Internet mailserver


Production serversIntranet serversInternet servers

- IT system resources requering IPS protection

- threat sources




Figure 8. IPS protection designing sample (determing the protections location)

Detecting and reacting on incidents related to security breaches (e.g. security breakthrough conditions) are the topics which should be taken into account in the security design. In reality there is no 100 percent effective protection and the situation that an intruder or a worm breaks into the network should be taken into account. A break-in can be done using unpublished security bug (so called zero-day exploit) or from within the network, bypassing the IPS protection. These topics should be analyzed in relation to the protected IT system resources and the security means which are planned to be deployed (security management tools included in them) and written in the incident management procedure.


© 2006 CLICO 15

4. Designing the Security Management Infrastructur e

Network security management include activities related to configuration (e.g. device parameters setting, policy creation), monitoring of security operation and detecting problems as well as reading, reporting and analyzing events logged (logs, alerts) and explaining security incidents. Maintenance and supervision of IPS protection in normal system operation conditions should be performed from central management system located in the network zone protected by a firewall (so called „Management VLAN”).

WAN

E-commerceserver

INTERNET

Mobileusers

Dial-up/ (W)LAN

Dial-up


Internetrouter

Officenetworks

Personal Firewall

Perimeter Firewall

WANrouter

Personal Firewall

SSL Accelerator

IPS

IPSmanagement

server

Firewallmanagement

server

Management_VLAN

Internal Firewall

IPS (2 VS) IPS (2 VS)

HQ domaincontroller

Applicationserver

Filesserver


Intranet mailserver

Databaseserver

RASserver

Internet mailserver


Production serversIntranet serversInternet servers




Figure 9. IPS protection designing sample (designing the protection’s management infrastructure)

Sample design of IPS protection with management system is presented in figure 9. In case of the systems with increased security requirements, management performance and reliability should be additionally taken into consideration (e.g. management station redundancy and spare control channels).


© 2006 CLICO 16

5. Elaborating IPS Security Policy

The IPS security policy describes the way in which different security devices perform monitoring and IT system resources protection tasks. The policy should describe, among others, the following factors:

− network traffic being protected,

− security reaction on attacks,

− the way of notifying administrators.

The IPS security policy form depends on the product chosen for project deployment. The security policy planning sample for IDP system has been presented below. The IDP policy consists of rules describing traffic being inspected (such as what are the client and the server of the application), what attacks should be detected in communication, what is the protection’s reaction and how administrators are notified of an incident as well as what the IDP device the policy rule applies to. Additionally IDP rules decide if particular rule finishes communication inspection or additional policy rules will be applied. For more information on this subject, refer to the product documentation [6].

Figure 10. IPS security policy using the example of Juniper Networks IDP


© 2006 CLICO 17

IDP security policy rules can have a broad scope of control or can detaily describe the network traffic being inspected. The rules of a broad control scope do not specify the communication being inspected (i.e. the Source and the Destination fields have „Any” value). This causes more load for security devices and the possibility to generate events of low importance for security administrators. The precise IDP rules specify communication type being inspected and the attacks detected. Generally it is reccommended that the precise security policy rules are used. However, it should be known that when using precise security rules it is easier to make mistakes (e.g. forgetting to include all the resources protected) and, as a cosequence, attacks can pass undetected. In such a case it is important to include the last rule, the so called clean-up rule which analyzes the network traffic which has not been analyzed by the previous rules.

The IDP protection’s reaction on the specific event depends on many factors (such as availability requirements of the resources being protected, location of the computer which has initiated the event, the threat scale, and so on). Generally, the attacks of high threat level, initiated from external networks against data-sensitive resources should be blocked. Alarming without blocking can be applied for the protection of mission-critical resources or for attacks initiated in local networks where administrators can control the users. Working in monitoring mode is also recommended in the initial stage of security deployment (i.e. during the policy tuning). In the field of notifying administrators about the event, the IDP security system allows several possibilities including writing information to log, displaying the event as an alarm, capturing specific number of packets before and after the event (e.g. in order to perform detailed analysis of the network traffic), sending message with SNMP Trap, Syslog and Email as well as running specific script or the application.

In the IDP solution, the security policy apart from protection against typical attacks from the network (exploit, DoS) can also include other security means used when needed, e.g.:

− detection of scanning and penetration attempts (Traffic Anomaly, Network Scanner Identification techniques),

− obstruction of scanning and reconnaissance (Network Honeypot mechanism),

− detection and blocking connection with trojan/backdoor applications (Backdoor Detection mechanism),

− detection of security breakthrough conditions (e.g. detecting ports newly opened in servers using Profiler mechanism),

− blocking packets with improper IP and MAC addresses (Anti-Spoofing),

− blocking of Spyware, Keylogger and other malicious applications (Server-to-Client type attacks),

− detection of unauthorized computers connected to the internal network (Profiler mechanism),

− supervision of complaying by employees the accepted security policy (Profiler mechanism),

− helping administrators in detecting systems and applications vulnerable to the security bugs (Profiler mechanism).


© 2006 CLICO 18

For early detection of incidents an important role plays deployment in the IDP security system means for recconnaisance techniques identification. Before attacks intruders usually make recognitions of attack targets in order to select proper tools. Enabling in the IDP a Network Honeypot mechanism is aimed to pass to intruders false information about system services available (usually the services specific for other system class are presented) and to show servers which in reality do not exist. When this mechanism is used, the intruder receives many false information obstructing the attack being performed (e.g. in the Windows server the services specific for Unix system are recognized). Operating system recognition using TCP/IP Fingerprint techniques is also much more difficult. Additionally, enabling in the IDP security policy Network Scanner Identification attack category allows for detection of tools used by intruders for scanning and penetration (e.g. Nessus, NMAP).

When an intruder successfully breaks in the system and installs a Backdoor, he can get unauthorized access to the system without conducting attacks. In the IDP security system there are two methods of counteracting such situations. Enabling the Protocol Anomaly Detection mechanism for IT system resources being protected (i.e. in the security policy rules the relevant Protocol Anomaly categories should be added) causes the network traffic being checked against conformity to the standards for specific protocols. Additionally, the Backdoor Detection mechanism should be configured. The mechanism detects an interactive communication specific for Backdoors using heuristic analysis techniques.

In case the attack against strategic IT system resources is detected, IDP system can block for specific time period, the access for IP addresses, the attack was conducted from. However, the settings should be well thought over before they are applied. This is very strong mechanism in the hands of administrator. Its use should be properly planned. First of all, such mechanism should not be used as a reaction to attacks, which could potentially be conducted from other IP addresses (e.g. attacks using IP Spoofing for forging addresses they are conducted from).

Apart from enabling proper protection mechanisms against attacks, it should also be noted, that the IDP security configuration be properly tuned in order to include specific IT system conditions, such as defining new attacks and events (e.g. blocking P2P applications specific for the particular country), as well as protection of services working on custom ports (i.e. defining new services and adding relevant security policy rules for them). An important role for proper IDP security management plays reducing false positive ratio and avoiding to detect events of low importance to the IT system security. This is performed by the analysis of the logs and alarms, and proper tuning of security policy rule content (e.g. removing from rules unnecessary attacks definitions). The recommendations for the IDP security system in this area can be found in document [5].


© 2006 CLICO 19

6. Designing Scalable and Reliable IPS Protections

The scalability of intrusion prevention systems means possibility to add more protection points and to improve performance and reliability of the existing protections without necessity to re-designing the network structure and its protections. The security design should ensure easy expansion of the security system by new elements and protection mechanisms in the area of its size (additional protection points), performance (higher network throughput and increased traffic) and reliability (redundancy configurations). When choosing a security product for project deployment, it is recommended that the IPS solution selected ensured protections flexibility i.e. possibility to adjust protections to changing network and computer systems conditions.

Availability of the IT system services working in a network environment depends on many different factors (e.g. network devices, communication links, security systems). Because of the specific of this environment, the particularly important element which influences system availability are security means applied for servers protection against undesirable activity of malicious users. The basic element of network security responsible for repulsing such attacks is the IPS system. Therefore it is important that the IPS is equipped with means protecting it against software and hardware failures. Configurations equipped with protection mechanisms against failures are called HA (high availability).

There are two basic architectures (categories) of HA systems:

– Active-Passive (also called as Hot-Standby) – the configuration consists of two or more IPS devices from which only one is active and the rest are spare machines used in case of failure of the active one,

– Active-Active – the configuration consists of two or more IPS devices linked as a cluster and balancing load between themselves.

Topics of designing the HA configuration will be described using the IDP solution as an example. The IDP security system allows for different HA configuration deployments working on the level of 2 and 3 OSI layers. Below, selected clustering configurations will be presented. The typical IDP cluster configuration has been presented in figure 11. Security devices work in L3 mode (Proxy-ARP or Router). When designing IDP security system in such configuration, it must be ensured that switches connected to the cluster properly handle multicast MAC addresses. This is so because a switch sends all the incoming traffic to all devices in the HA cluster. IDP devices communicate between themselves using heartbeat protocol and decide which of them will handle incoming sessions. The link for cluster synchronization (HA link) should be ensured on the L2 OSI level. The HA cluster presented in the figure can work in Active-Active or Active-Passive modes.


© 2006 CLICO 20

Figure 11. The IDP security devices cluster of Active-Active type

In case IDP security system is deployed as a complement to the firewalls working in the Active-Passive cluster mode, IDP devices can be transparently connected directly after firewall devices. Such configuration is shown in figure 12.

Figure 12. The IDP security devices cluster of Active-Passive type

switch

Man

agem

ent V

LAN

switch

IDP (1) IDP (2)

ETH3 ETH3

ETH2 ETH2

IDP Cluster (A/A) Proxy-ARP

ETH1ETH1

HA link

ETH0 ETH0

LAN

Man

agem

ent V

LAN

Firewall (1) Firewall (2)

FW Cluster (A/P)

switch

IDP (1) IDP (2)

ETH3 ETH3

ETH2 ETH2

IDP Cluster (A/P) Transparent

ETH1ETH1

HA link

ETH0 ETH0

LAN


© 2006 CLICO 21

The inspection of communication between network core and LAN access switches has specific requirements. Most often such links are used for VLAN networks (i.e. they are VLAN trunks). In case the IDP device work on the similar basis as network switch (Bridge mode), it is required that the STP protocol on the IDP devices is properly configured. Additionally VLAN interfaces should be set up and VR security module for each of them should be configured. It can be troublesome during security operation. For such uses more suitable are IDP systems working in Transparent mode, where the network traffic is inspected regardless of whether it is tagged (VLAN) or not. In case there is a trunk link between switches, configuration of VLAN and VR interfaces on the IDP device is not required. IDP device can then pass protocols other than IP allowing for proper communication between switches (e.g. STP, Rapid STP) as well as detection of links and devices failures. The status of links connected to the IDP is monitored through the peerPortModulator and in case of failure all IDP devices inspection interfaces belonging to the VR where the failure has been detected are closed. The goal of this is to quickly inform about the failure network devices working in the IDP vicinity.

Figure 13. The IDP security cluster working in Transparent mode

CORESWITCH

LAN SWITCH

CORESWITCH

LAN SWITCH

DATA CENTER

NETWORK CORE

Transparent ModepeerPortModulatorSTP pass-through

Man

agem

ent V

LAN

IDP (1) IDP (2)

ETH3 ETH3

ETH2 ETH2 ETH1ETH1

HA link (optional)

ETH0 ETH0


© 2006 CLICO 22

Ensuring a high level of protection for IT system resources against threats from the network and at the same time assuring the cost-effectiveness is possible by running many virtual security modules on the same physical IPS device (in the IDP solution virtual IPS modules are called virtual routers — VR). The VR modules working on the same physical device are separated from each other and can safely inspect different network segments. Sample uses of virtual security modules in IDP system have been presented in figures 14 and 15.

Figure 14. Using virtual IPS for protecting an Internet link

Figure 15. Using virtual IPS for protecting a servers farm

DMZ(2)

DMZ(1)

Firewall

Internet

Management VLAN

vr0 vr1 vr2 mgtIDP – 3 VR

eth0eth2

eth3

eth4

eth5

eth6

eth7

LAN

NETWORKCORE Core switch

LAN switch

vr0 vr1 vr0 mgtvr1

Servers farm

Management VLAN

VLAN Trunks

eth0mgt

eth0

ManagementVLAN

IDP – 2 VR

eth3

eth2

eth3

eth2

eth5 eth5

eth4 eth4


© 2006 CLICO 23

7. Security Incidents Handling

The IT system’s protections should be prepared for security breaches. Such situations are called security incidents. An incident is every adverse event which threatens confidentiality, integrity or availability of information resources, computer systems and networks used for delivering information. Every violation of security policy, accepted uses of policy or standard security practices is an incident. Incident samples include DoS attacks, loss of accountability, system damage, entering malicious code (such as virus, trojan/backdoor application) as well as unauthorized access to the system.

Incident response procedures contain guidelines for such topics as system preparation for incident handling, incident identification, stopping spreading incidents to other systems, elimination the vulnerabilities which have led to the incident as well as the analysis of events and drawing conclusions in order to avoid similar incidents in the future [3].

The most important goal of the incident handling is to maintain or to restore the continuity of business operation. Incident handling should include preventing the incident to spread to other systems (incident containment) and elimination the system vulnerability as well as blockade of threat sources (incident eradication). Administrator has at his disposal two basic incident handling methods: disconnecting the system from the network and restoring its proper operation (e.g. from the backup copies) or restoring the system operation without disconnecting it from the network.

There is a decision problem in systems of high availability requirements (i.e. mission-critical systems). Such systems can not be disconnected from the network for the incident is handled and its effects eliminated. In such cases the available access control means should be used in order to limit possibilities of spreading the incident to other systems and to block the source of the attack. For this purpose a dedicated firewall devices or filter modules included in other devices (e.g. routers, IPS) can be used. Both network protections and security management procedures should be prepared for such situations.

Incident handling begins from its detailed assessment — recognition of the security breach symptoms, identification of the incident type, identification and protection of evidences and reporting the event to the relevant persons or institutions. It is very important that on the stage of identification of the incident occurred in the computer network, the location of the attack source is found (e.g. IP address). Then the threat source can be separated from the IT system resources using available access control means. Identification of the type and location of the attack can be performed with intruder detection systems (IPS/IDS).

The potential threat source can also be the attacked system itself. The intruder or malicious code (e.g. worm) can use this system for escalating its attack against other IT system resources. In order to make spreading the incident impossible, the attacked system should be separated from the rest of the network using security means available. Quick response and precision during incident handling is of very high importance.


© 2006 CLICO 24

When selecting security means (firewalls) for the isolation of an attack source, the following guidelines should be taken into consideration:

− the attack source will be isolated from the IT system resources which may be potential target of the attack,

− security devices equipped with IPS/IDS will be used or devices for which IPS/IDS devices exist between them and the attack source. In order to explain all circumstances of the event, it is important that the incident source be continually monitored.

In order to limit the possibility to spread the incident to other systems, the attacked system should be treated as a potential threat source and when possible isolated from the rest of the IT systems. Action taken depends on security policy requirements for the attacked system. When confidentiality and integrity of the system are more important than its availability, the system can be disconnected from the network and restored in an off-line mode (e.g. from the backup). When the availability of the system is more important and the down-time is not acceptable, other action should be taken. In such scenario, the attacked system should be separated using security means available in the network, however access to the system services for important users (e.g. strategic company’s customers) should be maintained.

8. Designing the Acceptance Test Plan

The last stage of network security design is developing the plan of acceptance tests which will be performed after the protections are installed and configured. Based upon the positive result of the acceptance tests, the security system can be put into production. Included in the project the acceptance test plan should contain for each test the detailed description of it (e.g. used tools, input parameters) as well as expected (positive) test result. Acceptance tests consist of two parts: functional tests and tests of leak-proof and efficiency of protections.

Performing functional tests is aimed to check if all the protected resources and network services are available with required correctness and quality level (QoS). Usually it involves checking with standard software (for respective applications) the correctness and operation quality of all the services which according to the company’s security policy should be available.

The leak-proof and efficiency tests for protections are aimed to perform a reliable assessment of the network security level in the part defined in the project assumptions with regard to leak-proof and resistance to undesirable interference in protected systems operation. Verification of leak-proof and efficiency of the protection systems is often performed as penetration test with controlled break-ins simulation. The tests should be performed in the first place with common techniques and hacker tools [8]. For this purpose, the auditor’s workstation should be properly protected in order to avoid undesirable leakage of the test results to the Internet. The tests should be performed in close co-operation with administrators of systems being tested. It is particularly important when performing tests of system resistance to the destructive attacks, break-in simulations as well as tests of correctness the protection’s reaction on the attacks conducted. The security personnel involved in performing the tests should be properly trained.


© 2006 CLICO 25

Bibliography

[1] Brookshire D., "AV Diversification, Next Generation Network Defense", SANS Institute 2004.

[2] DISA, “Infrastructure Security Technical Implementation Guide”, US Defense Information Systems Agency 2003.

[3] FCC, “Computer Security Incident Response Guide”, US Federal Communications Commission 2001.

[4] IATFF, “Information Assurance Technical Framework”, IATFF 2002.

[5] Juniper Networks, „IDP Policy Building Primer, Building Scalable Policies with Juniper IDP”, Juniper Networks 2006.

[6] Juniper Networks, "Intrusion Detection and Prevention, Concepts & Examples Guide, Release 4.0r3", Juniper Networks 2006.

[7] NSA, “Defense in Depth - A practical strategy for achieving Information Assurance in today’s highly networked environments”, NSA 2000.

[8] Stawowski M., „Network security tests”, Arskom 1999.

[9] Stoneburner G., Hayden C., Feringa A., “Engineering Principles for Information Technology Security”, NIST 2004.

[10] Straub K.R., “Information Security Managing Risk with Defense in Depth”, SANS 2003.

[11] Zimmerman S.C., CERT Coordination Center, “Secure Infrastructure Design”, CERT 2001.

The Principles and Good Practices for Intrusion Prevention ... · The Principles and Good Practices...

Documents

Transcript of The Principles and Good Practices for Intrusion Prevention ... · The Principles and Good Practices...