Casey Robinson caseyr@gmail - GitHub PagesImplementation: LISYS 3 A-Brain: Acting Novelly 4...

Overview Artificial Immune Systems (AIS): Detecting Novelty A-Brain: Acting Novelly Conclusion

Artificial Intelligence and Novelty

Casey [email protected]

Division of Science and MathematicsUniversity of Minnesota, Morris

April 28, 2012


What is AI?

“The theory anddevelopment of computersystems able to performtasks that normally requirehuman intelligence, such asvisual perception, speechrecognition,decision-making, andtranslation betweenlanguages” [Oxford EnglishDictionary]This definition includes bothlearning and static systems

Photo credit Norris Labs,(norrislabs.com)


The Problem With Novelty

Static systems often cannotcope with factors notconsidered during theirdesignLearning systems requirerepeated exposure to data toencode an appropriatereactionSpurious or very infrequentevents may have moreimportance than those seenmore frequently, but areharder to learn about

Top photo credit: Adam Hart-Davis

Bottom: rickkirvan.net46.net


1 Overview

2 Artificial Immune Systems (AIS): Detecting Novelty

3 A-Brain: Acting Novelly

4 Conclusion


1 Overview

2 Artificial Immune Systems (AIS): Detecting NoveltyOverview of AISStrengthening AISImplementation: LISYS


4 Conclusion


Overview

Based upon a model ofbiological immune systemsDetects behaviors notconsidered "normal"This can be useful in thecontext of network security,where detecting novelattacks would have greatvalue

Antigens

Antibody

Antigen

Antigen-binding site


Detectors

Detectors fit therole of antibodiesin an immunesystemDetectors arebitstrings,compared totarget data

detector

ACCEPT REGENERATE

Randomly generatedetector string

If detector matches self,regenerateotherwise accept

self

Figure: From [Hofmeyr-Forrest, 2000]


Coverage and Generalization

Adequate Fitting

It is impossible to train anAIS on absolutely everythingit should view as selfAn overzealous fit to thetraining data will yield an AISwhich generates a lot offalse positivesA system which is tooloosely fit, by contrast, willoften fail to recognizenon-self elements

What the AISwill see during training

All data comprising the self

Figure: In the above diagram, the reddashed circle represents the result ofunderfitting, and the green circlerepresents the result of overfitting.


1 Overview



4 Conclusion


r -Contiguous Bits Matching

Improving Match Rates

Chances of a completematch between a detectorand a bitstring is very low forlarge detectorsr -chunk matching schemevastly increases thelikelihood of a match

1001011011010100

Detector

Target

Figure: Two different bitstrings,which have a match for r=4


Unreachable Patterns

Even usingr -chunksdetection, somepatterns cannotbe detectedRemoving thecontiguousconstraint willproduce far toomany matches

holes

holes



Permutation Masks

Permutation masks reorder data in a predictable way, allowingdetection of some such features

non-permuteddetector set

permuted detectorset 1

permuted detectorset 2

detection across all sets



Permutation Mask Example

1 2 3 4 5 6 7 8

1x11x1xx

x1111xxx

Original

Permuted

Figure: Application of a permutation mask 2-3-4-6-1-5-8-7


1 Overview



4 Conclusion


Overview

Developed by StevenHofmeyr and StephanieForrestChecks packets beingtransmitted over a networkDoes not use timing dataMultiple detectors mustmatch a packet in order forthe packet to be considerednon-self

The Internet

LISYS

LocalMachine

LocalMachine

LocalMachine

LocalMachine

LocalMachine


LISYS Packet Representation

Checking every packet sent over a network would invoke a lot ofoverheadLISYS examines a compressed representation of SYN packets,sent when a connection is being established between twomachinesLISYS only decides whether a connection should be allowed

49-Bit Packet Representation

Ê Ñ Ó Ø Ö × ×

Ö ØLocal address

0-7 8-39Remote Address Compressed Port

41-48Incoming/Outgoing

Bit


Experimental Setup

Data set was 15,000 packets – roughly 8 days of normal trafficLISYS developed self-model by examining roughly half of this setDue to the fact that the same machines may connect manytimes, only 131 unique strings in training setBoth normal detectors and detectors using permutation maskswere generatedAfter training, remaining half of data set with 400 packets of“attack” data shown


Results

Permuted DetectorsMatched almost all attackdata, some had better falsepositive rates than others

Packet Representation

Local:Remote:IO:Port

DefinitionsFalse positive –Self-data which thesystem classified asnon-selfTrue positive – Attackdata which the systemclassified as non-self.

0 5 10 15 20 25

False Positives

0

100

200

300

400

Tru

ePos

itive

s

permutationsunpermuted

Figure: Based on [Balthrop et al., 2002]


Results

Full-Length Detectors

Matched most attack data,but also a high proportionof non-attack data

r -Chunk DetectorsMatched even more attackdata, tended to have fewerfalse positives (varying withvalue of r )

0 5 10 15 20 25

False Positives

0

100

200

300

400

Tru

e P

osit

ives

r-chunk detectorsfull length detectors

Figure: From [Balthrop et al., 2002]


Analysis

Small r ResultsGood performance onsmall values of r wasdue to how dynamic /static IP addresseswere assigned in theexperimentStatic IP addresseswere (0..127).x.x.x,while dynamic IPaddresses were(128..255).x.x.x

0 5 10 15 20 25

False Positives

0

100

200

300

Tru

e P

osit

ives

r-chunk detectorsfull length detectors

Figure: Results after removal of "magic bit."From [Balthrop, et al., 2002].


General AIS Results

Strengths of AIS

Given sufficient training onany self, AIS can recognizenon-self rather efficientlyMakes no assumptionsabout the state of non-self,so both familiar and novelnon-self elements aredetected

Weaknesses of AISCannot determine whysomething is non-self, onlythat it is non-selfEnsuring an appropriatedegree and type ofgeneralization is very difficult

Other Applications of AIS

Industrial settingsMalware detectionMore?


1 Overview

2 Artificial Immune Systems (AIS): Detecting Novelty


4 Conclusion


A-Brain: A General Problem Solving Algorithm

Extends the concept of evolutionary programmingDevelops solutions to problems, keeps a record of solutions forproblems already seenIts ability to recognize solved problems is fragile – depends onaccurate user input


A-Brain Operation

Known Problem 1

New Problem

Problem Classifier

Request trainingexamples from user

Solver for problem 1



Solver fornew problem

Trainer

Recognized problems

New problem

Known Problem 2

Known Problem 3

Figure: From [Oltean, 2007]

For further information on A-Brain, see the original text [Oltean2007], or the paper associated with this presentation.


Conclusions

Two Sides of the Same CoinAIS good at recognizingnovelty but cannot address itA-Brain capable ofresponding to novelty, butbad at recognizing itBoth aspects are important,but neither one more so thanthe other

Importance of Novelty

An ability to appropriately handlenovel data is key to creating asystem which we would identifyas “smart”


Thank You

Comments?Questions?


References

References

1 Stephen Hofmeyr and Stephanie Forrest, Architecture for anArtificial Immune System, 2000.

2 Mihai Oltean, A-Brain: A General System for Solving DataAnalysis Problems, 2007.

3 Balthrop et al, Coverage and Generalization in an ArtificialImmune System, 2002.


Detectors, cont

Short DetectorsDetectors of length N can be used to examine strings of length ≥ N.

Target 0 1 0 0 1 0 1No match 1 0 0 1

Match 1 0 0 1No match 1 0 0 1No match 1 0 0 1

Casey Robinson caseyr@gmail - GitHub PagesImplementation: LISYS 3 A-Brain: Acting Novelly 4...

Documents

Transcript of Casey Robinson caseyr@gmail - GitHub PagesImplementation: LISYS 3 A-Brain: Acting Novelly 4...