O uso da Transformada de Haar na Detecção de …ccappo/SBSEG2013-SLIDES.pdf · O uso da...

O uso da Transformada de Haar naDetecção de Anomalias no Tráfego Web

C. Cappo1 R. C. Nunes2 B. Mozaquattro2 A.Kozakevicius2 C. Schaerer1

1Facultad Politécnica,Universidad Nacional de Asunción, Paraguay

2Centro de TecnologíaUniversidade Federal de Santa María, RS, Brasil

XIII Brazilian Symposium on Information and Computer Systems Security

IntroductionOur approach to detect anomalies in web applications

Experiments and ResultsConclusions and future Work

Outline

1 IntroductionMotivationAnomaly detection

2 Our approach to detect anomalies in web applicationsMain characteristicsWavelet Transform TheoryWavelet Algorithm for attack Detection

3 Experiments and ResultsDataset & AttacksResults

4 Conclusions and future Work

C. Cappo,R.C.Nunes, B. Mozaquattro, A. Kozakevicius and C. Schaerer 2



MotivationAnomaly detection

Outline









Motivation

Internet has become a habitual tool used by millions ofpeople in the world.The use of web applications, such as, blogs, news, socialnetworks, webmails, e-commerce, among may others, hasbecome conventional.Protecting these applications from attacks is a criticalissue.

The number of new vulnerabilities discovered in 2012 were5291 and web-based attacks increased by almost a third in2012 (according to Symantec Internet Security ThreatReport, 2013 - Vol 18)

One form of protection is to use Intrusion DetectionSystem (IDS).There are two main approaches in detection algorithmsIDS design: signature-based and anomaly-based.We focus on the design of anomaly-based detectionalgorithms.





Outline









Anomaly-based approach

The analysis is based on the observation of anysubstantial variation of any specific characteristic withrespect to the commonly determined behavior.A significant deviation from usual behavior is consideredan anomaly, and so an attack.Does not need the knowledge of previous attack pattern.Can potentially detect novel attacks.





Anomaly Detection in Web Application

In the context of web application this approach has thefollowing advantages:

No requirement of a priori knowledge of theweb-application.Capacity of self adaptation to periodic maintenance of theweb applications in focus.Polymorphic and unknown attacks detection capacity (ex.zero-day attack)Custom-developed web applications protection skill.

We focus in anomaly-based algorithms to detect attackagainst web applications.




Main characteristicsWavelet Transform TheoryWavelet Algorithm for attack Detection

Outline









Characteristics(1)

The detector analyzes the HTTP requests sending to the web application

[IP] - - [TS] "GET /page.php?p=calAcad HTTP/1.1" ..[IP] - - [TS] "GET /page.php?p=allnews HTTP/1.1" ..[IP] - - [TS] "GET /page.php?p=trabajo HTTP/1.1" ..[IP] - - [TS] "GET /page.php?p=ingeInfo HTTP/1.1" ..[IP] - - [TS] "GET /page.php?p=ingeInfo HTTP/1.0" ..[IP] - - [TS] "GET /page.php?p=mapsite HTTP/1.1" ..[IP] - - [TS] "GET /page.php?p=admision HTTP/1.1" ..[IP] - - [TS] "GET /page.php?p=ingeInfo HTTP/1.1" ..[IP] - - [TS] "GET /page.php?p=materias HTTP/1.0" ..[IP] - - [TS] "GET /page.php?p=examenes HTTP/1.1" ..

The data analyzed for the anomaly detection is the URL Query String of theHTTP request.





Characteristics(2)

The data model is based in the character distribution of theURL Query String.Our method requires only a few normal data for frequencyenhancement. The principal detection algorithm is basedonly in current data. The principal hypothesis is thatattacks perturbs significantly the frequency of somecharacters.We apply the bidimensional Discrete Wavelet Transform(DWT), particularly the Haar Wavelet Transform, to detectthe anomalies in character frequency distribution.





Modeling the anomaly using the character distribution

A window analyzed without attacks

50

100

150

200

250 0 50

100 150

200 250

0

2

4

6

8

10

12

14

16

18

Frequency

(a)

ASCII

HTTP Request

Frequency 0

50

100

150 0 50 100 150 200 250

AS

CII

HTTP Request

(b)

0

2

4

6

8

10

12

14





Modeling the anomaly using the character distribution

A window analyzed with two attacks

50

100

150

200

250 0 50

100 150

200 250

0

2

4

6

8

10

12

14

16

18

Frequency

(a)

ASCII

HTTP Request

Frequency 0

50

100

150 0 50 100 150 200 250

AS

CII

HTTP Request

(b)

0

2

4

6

8

10

12

14





Outline









Wavelets - Introduction

The wavelet transform extracts information from theanalyzed data in different resolution levels.Describes a signal in terms of a coarse overall shape plusa family of details.In the bidimensional case, the input data is given as amatrix and the 2D Discrete Wavelet Transformationconsists in performing the 1D wavelet transform in all rowsand then in all columns.





One-Dimensional Wavelet Transform (TW1D)

The TW1D is stated as following: considering the initialinput data a vector cJ,s, s = 0, ...,MJ − 1 at the finest level J,with MJ = 2J points, we have the following relations for plevels, when j = J, J − 1, ..., J − p :

cj−1,i =2N−1∑k=0

Lkcj,2i+k, i = 0, ...,Mj−1 − 1, (1)

dj−1,i =2N−1∑k=0

Hkcj,2i+k, i = 0, ...,Mj−1 − 1, (2)

DefinitionConsidering the orthonormal family of Wavelet Functions, the TW1Dis defined by high pass and low pass filters of size 2N, L and Hrespectively.





One-Dimensional Wavelet Transform (TW1D)

Vector cj−1,i contains the coarser information and thevector dj−1,i contains the wavelets coefficients, both withMj−1 = Mj/2 points.We consider using the Haar wavelet family (N = 1). Thefilters are given by L0 = 1√

2, L1 = 1√

2, H0 = 1√

2and

H1 = − 1√2.

We use the Haar transform because:Simple and fast algorithmsWithout boundary problemsIdeal compact support (shortest support) considering theimportance of preserving the anomalies location.





TW1D example

-10123

-1 0 1

Value

Samples interval

Original Signal

-0.4

0.0

0.4

-1 0 1

Value

Approximation coefficients - 1 level

-0.1

0.0

0.1

-1 0 1

Wavelets coefficients - 1 level

-0.4

0.0

0.4

-1 0 1

Value

Approximation coefficients - 2 level

-0.1

0.0

0.1

-1 0 1

Wavelet coefficients - 2 nivel

-0.4

0.0

0.4

-1 0 1

Value

Samples interval

Aproximation coefficients - 3 nivel

-0.1

0.0

0.1

-1 0 1

Samples interval

Wavelet coefficients - 3 nivel





Algorithm 1: DecompositionInput : C[1..M]

1 while M > 1 do2 DecompositionStep(C)3 M ← M

24 end

5 return

Algorithm 2: DecompositionStepInput : C[1..M]

1 C′ ← 02 for i← 1 to M

2 do3 C′[i]← (C[2i− 1] + C[2i])/

√2

4 C′[ M2 + i]← (C[2i− 1]− C[2i])/

√2

5 end6 C ← C′

7 return





Bi-Dimensional Wavelet Transform (TW2D)

Algorithm 3: TW2DInput : X[1..h, 1..h]

1 while h > 1 do2 for row← 1 to h do3 DecompositionStep(X[row, 1..h])4 end5 for col← 1 to h do6 DecompositionStep(X[1..h, col])7 end8 h← h

29 end

10 return

X

L

H

L

H

L

H

2 c

TW1DPor linhas

TW1D por colunas

2 d

cc

dc

2

2

2

2

cd

dd

Figura : TW2D scheme for onetransformation level





TW2D example





Thresholding Operation

This operation is used to select the most significantwavelet coefficient and to discard irrelevant informations.Usually the threshold operation is used for signaldenoising.We use the threshold value λ as limit of normal waveletcoefficients.When |dk(j)| > λ, the position k associated for the level j isconsidered anomalous.For compute the threshold value we use the UniversalThreshold, given by λ = σ

√2log(T), where σ and T are the

standard deviation and number, respectively, of the waveletcoefficients.





Outline









Data Model

The character frequency associated to data collected fromthe web server is organized in the input matrix.The input matrix is defined by Xrc, 0 ≤ r ≤ 255 and1 ≤ c ≤ m, where the value m is the number of therequests. For experiments we use m = 256.

Request (1-m)

ASC

II C

har

(0-2

55)

0

255

1 m. . .

.

.

.

f

. . .

.

c

r

frequency f of character r in the request c





Detection with TW2D

A analyzed window with one attack

0

50

100

150

200

250 0 50

100 150

200 250

0

2

4

6

8

10

12

14

16

18

Frequency

a)

ASCII

HTTP Request

Frequency 0

50

100

150

0 50 100 150 200 250

AS

CII

HTTP Request

b)

0

2

4

6

8

10

12

14

A1

A TW2D of the analyzed window above

0

50

100

150

200

250 0 50

100 150

200 250

0

2

4

6

8

10

12

14

16

ab

s(C

oe

ffic

ien

t)

a)

ASCII

HTTP Request

ab

s(C

oe

ffic

ien

t)

0

50

100

150

200

250

0 50 100 150 200 250

AS

CII

HTTP Request

b)

0

2

4

6

8

10

12

14

(cc) (cd)

(dc) (dd)





Anomaly Detection Scheme





Pre-detection

Weight computation for each character ci, i = 0..255 for kpreprocess windows. fj(ci) is the frequency of character ci inwindow j.

p(ci) =

{ 1∑kj=1 fj(ci)

,∑k

j=1 fj(ci) > 0

1 ,∑k

j=1 fj(ci) = 0i = 0..255 (3)





Detection: Anomaly Detection Algorithm

Frequency enhancement phase according to weights computedin predetection phase.

f ∗(ci) =

{f (ci) + p(ci) ∗ CTE , f (ci) > 0

0 , f (ci) = 0 i = 0..255 (4)

The TW2D generates four blocks of coefficients: approximationblock (cc) and 3 coefficients blocks (cd, dc, dd).When the wavelet coefficient (of any block) is greater than λ,then its associate request is considered anomalous.λ is computed for each coefficient block using the UniversalThreshold Value λ = σ ·

√2log(T). In this work we compute the

σ approximation as mean of the absolute deviation from themedian (named ad). σ = 1

N

∑Ni=1 |di − med(G)|, i = 1 . . . T where

med(G) is the median of wavelets coefficients |di| > 0 of block G.





Anomaly Detection Algoritm

The algorithm is summarized belowInput The matrix X;

Step 1 Frequency Enhancement;Step 2 Apply the TW2D of X one level;Step 3 For each subband (cd, dc, dd) to compute a

threshold limit λ ;Step 4 For each subband (cd, dc, dd) to mark the position

x, y if |dxy| > λ ;Step 5 If the position x, y was marked in almost two

subband then it correspond to attack.





Effect of Frequency enhancement (example)Without enhancement With enhancement

50

100

150

200

250 0 50

100 150

200 250

0

5

10

15

20

25

Frequency

ASCII Character

Request HTTP

Frequency

50

100

150

200

250 0 50

100 150

200 250

0

5

10

15

20

25

30

Frequency

ASCII Character

Request HTTP

Frequency

Pos Attack10 /page.php?p=%2e%2e%2f%2e%2e%2f/../../../../../../etc/passwd25 /page.php?p=%2e%2e%2f%2e%2e%2f/../../../../../etc/passwd36 /page.php?p=xxxxxxxxxxxxxxxxxxx74 /page.php?p=../../../../../../../../../../../etc/passwd%00100 /page.php?p=http://www.manchenumerique.fr/voeux2008/rss.txt??212 /page.php?p=../../../../../../../etc/passwd%00246 /page.php?p=../../../../../../../etc/passwd%00




Dataset & AttacksResults

Outline









Dataset for experiments

The dataset contains queries sent by clients to a webserver in log format, for instance:

170.51.19.9 - - [11/Jan/2010:20:41:19 -0300] "GET /page.php?p=calAcad HTTP/1.1" 200.The data collected corresponding to three months webtraffic of Polytechnic School web serverThe total number of request was 59248 and 232 the totalnumber of processed windows.The attacks were manually inserted in the dataset andincluded the following attacks: Directory Traversal,Code-Red and Cross Site Scripting attack (XSS),FileInclusion, SQLInjection and OSInjection .





Kind of attacks inserted in the database

Attack Example Quant.

FileInclusion /page.php?p=http://www.manchenumerique.fr/voeux2008/rss.txt?? 1

CodeRed /page.php?p=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 2

Directory Traversal /page.php?p=../../../../../../etc/passwd%00 8

XSS /page.php?p=<scr<script>ipt>alert(document.cookie)</script> 5

SQLInjection /page.php?p=gd_index and 1 = 1 5

OSInjection /page.php?p=/bin/ping 1

Total 22





Outline









Comparison results between with and withoutenhancement

Attack Total Without Enhancement With EnhancementFileInclusion 1 0 1CodeRed 2 2 2Directory Traversal 8 8 8XSS 5 2 5SQLInjection 5 0 5OSInjection 1 0 1TP 22 12 22FP 0 0 4Precision(P) 100% 85%Recall(R) 55% 100%FMeasure 71% 92%

FP = False Positive TP = True Positive FN= False Negative

P = TPTP+FP

R = TPTP+FN

FMeasure = 2∗R∗PR+P

Number of windows for predetection phase: 4 = 1024 requests





Comparison results with others anomaly algorithms

The algorithms considered here for comparison require normaldata for training phase.

Attack Total TW2D with enhancement 6BIN MD NGRAMFileInclusion 1 1 0 1 1CodeRed 2 2 2 0 2Directory Traversal 8 8 8 6 8XSS 5 5 0 5 5SQLInjection 5 5 0 5 5OSInjection 1 1 0 1 1TP 22 22 10 20 22FP 0 4 26 21 231Precision(P) 85% 28% 48% 9%Recall(R) 100% 46% 91% 100%FMeasure 92% 34% 63% 16%

6BIN: Person χ2 test [Kruegel and Vigna 2003] [Kruegel et al. 2005]

MD: Mahalanobis distance [Wang and Stolfo 2004]

NGRAM: Algorithm based in ngram analysis [Ingham and Inoue 2007]. We had considered 2-gram to10-gram and we put best results here. A request is anomalous if it have less than 95% normal ngrams.

Number of windows for predetection phase: 4 = 1024 requests. For others algorithms we use 1024 requestsfor training phase.




Conclusions and future Work

We have showed an algorithm based in Haar WavaletTransform with a frequency enhancement preprocess.The threshold used in the algorithm attack detection isadapted to analyzed data. This is a local adaptivethreshold.The frequency preprocess phase permits to identify moresubtle attacks. This improves the sensor performance.Our method outperformed other traditional anomalymethods that analyze character frequency distribution.In a future work, we will analyze the behavior of proposedalgorithm in other databases. We will extend the analysisto HTTP POST request and HTTP header fields. Finallywe will test our algorithm with other sort of web attack.




Questions?Thanks for your attention!!

Cristian Cappo ([email protected] )


O uso da Transformada de Haar na Detecção de …ccappo/SBSEG2013-SLIDES.pdf · O uso da...

Documents

Transcript of O uso da Transformada de Haar na Detecção de …ccappo/SBSEG2013-SLIDES.pdf · O uso da...