[email protected], 1 Toward Systems With a Will to Live SORNS Self Organizing Resilient Network...

[email protected], 1

Toward Systems With a Will to Live

SORNSSelf Organizing Resilient Network Sensing

3Mar2010

This project is sponsored by the Department of Homeland Securityunder contract D10PC20039. The content of the material

contained herein does not necessarily reflectthe position or policy of the Government, and

no official endorsement is implied.


General Current Situation

AdversarialAgent (AA)

Adversarial Domain (AD)

AA AA





AdversarialCommunities AD AD

AA AA





AdversarialCommunities

SecurityAgent (SA)

Security Domain (SD)

SecurityCommunities SD SD

SA SA

AD AD

AA AA






SecurityAgent (SA)


SecurityCommunities

Relatively StaticSecurity and System

Artifacts (A)

A

A

AA A

AA A

A

A

A

A

AA

A

A

Static artifacts are systems with and without security measures, updated occasionally.

AD AD

AA AA

SD SD

SA SA

Static Artifact






SecurityAgent (SA)


SecurityCommunities


Artifacts (A)

A

A

AA A

AA A

A

A

A

A

AA

A

A

Dynamic attack includes human and systemic adaptive control preying upon fixed artifact defenses.


AD AD

AA AA

SD SD

SA SA

Static ArtifactDynamic Attack






SecurityAgent (SA)


SecurityCommunities


Artifacts (A)

Static ArtifactDynamic Attack

Dynamic attack includes human and systemic adaptive control preying upon fixed artifact defenses.


AD AD

AA AA

SD SD

SA SA


StaticSystem

DynamicAdversary


AsymmetriesAdversary is a natural system, security strategy is an artificial system

Adversary leads with innovation and evolution

Adversary self-organizes as a dynamic system-of-systems

Three Dynamic Self Organizing System-of-System Security Patterns

Pattern employment on the SORNS project

… up next …

Inspirational Patternsfrom natural systems that effectively process

noisy sensory input from uncertain and changing environments


Pattern: Bow Tie Processor (assembler/generator/mediator)

Context: Complex system with many diverse inputs and many diverse outputs, where outputs need to respond to many needs or innovate for many or unknown opportunities, and it is not practical to build unique one-to-one connections between inputs and outputs. Appropriate examples include common financial currencies that mediate between producers and consumers, the adaptable biological immune system that produces proactive infection detectors from a wealth of genetic material, and the Internet protocol stack that connects diverse message sources to diverse message sinks.

Problem: Too many connection possibilities between available inputs and useful outputs to build unique robust, evolving satisfaction-processes between each.

Forces: Large knot short-term-flexibility vs small knot short-term-controllability and long-term-evolvability (Csete 2004); robustness to known vs fragility to unknown (Carlson 2002).

Solution: Construct relatively small “knot” of fixed modules from selected inputs, that can be assembled into outputs as needed according to a fixed protocol. A proactive example is the adaptable immune system that constructs large quantities of random detectors (antigens) for unknown attacks and infections. A reactive example is a manufacturing line that constructs products for customers demanding custom capabilities.

Millions of random infection detectors generated continuously by fixed rules and modules in the “knot”

Evolve three fixed V-D-J gene-segment libraries

Fixed-rule VDJ assembly with random interconnects

Random high variety outputwith VDJ + VJ assemblies

Available high varietygenetic DNA input

V: 123 Variable segments

D: 27 Diverse segments

J: 6 Joining segments

increases to

~109 varieties withaddition of randomnucleotide connections between VDJ & VJ joinings

~106 VDJ+VJ possible antigen detectorshapes

V1

D1

Vn

Vr JrDrr r

123 Vs

27 Ds

6 Js

1 random from each+ random connect

Dn

JnJ1


Adaptable Immune SystemBow-Tie Antigen-Detector Generator

(generic agile-system architectural-concept diagram)

detector sequence n

shortchain

longchain

detector sequence n+1

shortchain

longchain


shortchain

longchain

cellbody

Y detector antibodyB-Cell

V--D--J V--J

123 V segments 6 J segments27 D segmentsrandom

nucleotides

Infrastructure evolution

Detector assembly

Module pools and mix evolution

Module inventory condition

Combine two assembliesAdd random nucleotides

Use one each V-D-JUse one each V-J

Infrastructure

Assembly Rules

IntegrityManagement

Active

Passive

genetic evolution

bone marrow and thymus

genetic evolution

??repair mechanisms??

(rules evolution)

(system assembly)

(module evolution)

(inventory condition)

Module Pools


Adaptable Immune SystemBow-Tie Antigen-Detector Generator

(generic agile-system architectural-concept diagram)

detector sequence n

shortchain

longchain


shortchain

longchain


shortchain

longchain

123 V segments 6 J segments27 D segmentsrandom

nucleotides

Infrastructure evolution

Detector assembly

Module pools and mix evolution

Module inventory condition

Combine two assembliesAdd random nucleotides

Use one each V-D-JUse one each V-J

Infrastructure

Module Pools

Assembly Rules

IntegrityManagement

Active

Passive

genetic evolution

bone marrow and thymus

genetic evolution

??repair mechanisms??

(rules evolution)

(system assembly)

(module evolution)

(inventory condition)

cellbody

Y detector antibodyB-Cell

V--D--J V--J

systemic integrity-management processes


Context: A complex system or system-of-systems subject to attack and infection, with low tolerance for attack success and no tolerance for catastrophic infection success; with resilient remedial action capability when infection is detected. Appropriate examples include biological organisms, and cyber networks for military tactical operations, national critical infrastructure, and commercial economic competition. Problem: Directed attack and infection types that constantly evolve in new innovative ways to circumvent in-place attack and infection detectors. Forces: False positive tradeoffs with false negatives, system functionality vs functionality impairing detection measures, detectors for anything possible vs added costs of comprehensive detection, comprehensive detection of attack vs cost of false detection of self. Solution: A high fidelity model of biological immune system antibody (detection) processes that generate high quantity and variety of anticipatory speculative detectors in advance of attack and during infection, and evolve a growing memory of successful detectors specific to the nature of the system-of-interest.

Speculative generation and mutation of detectors recognizes new attacks like a biological immune system

Pattern: Proactive Anomaly Search


Context: A decision maker in need of accurate situational awareness in a critical dynamic environment. Examples include a network system administrator in monitoring mode and under attack, a military tactical commander in battle, and the NASA launch control room.Problem: A very large amount of low-level noisy sensory data overwhelms attempts to examine and conclude what relevance may be present, most especially if time is important or if sensory data is dynamic. Forces: amount of data to be examined vs time to reach a conclusion, number of ways data can be combined vs number of conclusions data can indicate, static sensory data vs dynamic sensory data, noise tolerated in sensory data vs cost of low noise sensory data.Solution: Using a bow-tie process, each level looks for a specific finite set of data patterns among the infinite possibilities of its input combinations, aggregating its input data into specific chunks of information. These chunks are fed-forward to the next higher level, that treats them in turn as data further aggregated into higher forms of information chunks. Through feedback, a higher level may bias a lower level to favor certain chunks over others, predicting what is expected now or next according to an emerging pattern at the higher level. Each level is only interested in a small number of an infinite set of data-combination possibilities, but as aggregation proceeds through multiple levels, complex data abstractions and recognitions are enabled.

Four level feed forward/backward sense-making hierarchy modeled on visual cortex

Pattern: Hierarchical Sensemaking

BIS Architecture(Biological Immune System)


Antibody Creation & Life Cycle

1

2

3

4

5

6

Diagram modified from (Hofmeyr 2000).

General antibody life cycle: creation, false-positive testing, deployment efficacy or termination, mutation improvement, and long-term memory.

1. Candidate antibody semi-randomly created.2. Tolerization period tests immature candidates for false-positive matches.3. Mature & naïve antibodies put into time limited service.4. Activated (B-cell) antibodies need co-stimulation (by T-cells) to ensure “improvement” didn’t produce

auto-reactive result, non-activated & non-co-stimulated candidates die when time limit ends.5. Highest affinity co-stimulated antibodies are remembered for

time-limited long term (eg, many years, decades). 6. Co-stimulated antibodies are cloned with structured mutations,

looking for improved (higher) affinity scores.


Antibody Creation & Life Cycle

1

2

3

4

5

6


General antibody life cycle: creation, false-positive testing, deployment efficacy or termination, mutation improvement, and long-term memory.

1. Candidate antibody semi-randomly created.2. Tolerization period tests immature candidates for false-positive matches.3. Mature & naïve antibodies put into time limited service.4. Activated (B-cell) antibodies need co-stimulation (by T-cells) to ensure “improvement” didn’t produce

auto-reactive result, non-activated & non-co-stimulated candidates die when time limit ends.5. Highest affinity co-stimulated antibodies are remembered for

time-limited long term (eg, many years, decades). 6. Co-stimulated antibodies are cloned with structured mutations,

looking for improved (higher) affinity scores.

Self nonself discrimination: A universe of data points is partitioned into two sets – self and nonself. Negative detectors cover subsets of non-self. From (Esponda 2004)

Shape/Pattern Space ~109

SORNS ApplicationSelf Organizing Resilient Network – Sensing


Reconfigurable Pattern ProcessorReusable Cells Reconfigurable in a Scalable Architecture

Up to 256 possible features can be “satisfied” by all so-designated byte values

Cell-satisfaction activation pointers

Independent detection cell: content addressableby current input byte

If active, and satisfied with current byte, can activate

other designated cellsincluding itself

Individual detection cells are configured into detectors by linking activation pointers.

All active cells have simultaneous access to current data-stream byte

an unbounded number of detector cells configured as dectectors can extend indefinitely across multiple processors

Cell-satisfaction output pointers


Reconfigurable Pattern ProcessorReusable Cells Reconfigurable in a Scalable Architecture

Up to 256 possible features can be “satisfied” by all so-designated byte values

Cell-satisfaction activation pointers

Independent detection cell: content addressableby current input byte

If active, and satisfied with current byte, can activate

other designated cellsincluding itself

Individual detection cells are configured into detectors by linking activation pointers.

All active cells have simultaneous access to current data-stream byte

an unbounded number of detector cells configured as dectectors can extend indefinitely across multiple processors

Cell-satisfaction output pointers

Enables High Fidelity Modeling


Intra-SORN-SCollaboration

Inter-SORNCollaboration

Human Interface

HumanControl

//

// Inter-SORNCollaboration

Level 1Agents

Level 2Agents

Level 3Agents

Level 4Agents

Human Interface

HumanControl

EP AttackDetectors

EP InfectionDetectors

Intra-SORN-SCollaboration

EP AttackDetectors

EP InfectionDetectors

SORN-S Hardware Device

SORN-S ArchitectureSelf Organizing Resilient Network - Sensing

Architecture anticipates collaboration with other SORN by Level 3 agents

Multi-level architecture refines sensory input through learning and sensemaking hierarchy, supports remedial action agents (human/automated) with succinct relevant information.

Notes: • For all-level hierarchical network-agent architecture general concept see (Haack 2009)• For hierarchical feed-forward/backward pattern learning, prediction, and sense-making see (George 2009).• For all-level hierarchical learning of causal patterns spread as time-sequence events see (Hawkins 2010).


Level 1 & 2 Agent: Detector Creation & Learning Architecture

L2 Agent

OtherL2 Agent

OtherL2 Agent

1

2

3

4

5

6

7


General L1 detector life cycle: creation, false-positive testing, deployment efficacy or termination, mutation improvement, and long-term memory.

1. Candidate fuzzy detector semi-randomly created.2. Tolerization period tests immature candidates for false-positive matches.3. Mature & naïve candidates put into time limited service.4. Activated (B-cell) candidates wait for co-stimulation (by T-cells) to ensure “improvement” didn’t

produce auto-reactive result, non-activated & non-co-stimulated candidates die when time limit ends.5. Highest scoring co-stimulated candidates are remembered for

time-limited long term. 6. Co-stimulated candidates are cloned with structured mutations,

looking for improved (higher) activation scores. 7. Level 2 Agent insertion of activated candidates from other end-

points, and Level 2 Agent distribution of activated candidates to other end points.


Determining an AIS Approach: According to Jon TimmisAmelia Ritahani Ismail. 2008. On the Use of Modelling and Simulation for Granuloma Formation. Department of Computer Science, University of

York, Final Qualifying Dissertation, Supervisor: Prof. Jon Timmis. www.cosmos-research.org/wiki/images/0/05/FINAL_QD.pdf

The Structure of AIS – de Castro & Timmis (2002) proposed a layered framework for engineering AIS that takes the application domain as a starting point followed by three design layers that form the structure of AIS (refer to figure 2.6). These layers are:• the representation of the components of the systems and known as shape space• a set of mechanisms to evaluate the interactions of the individuals with the environment known as the

affinity measures • the use of the algorithm (such as clonal selection, negative selection, immune network) which governs

the behaviour of the systems de Castro & Timmis (2002) argue that the basis of every system is the application domain, therefore the way in which the components of the systems will be represented has to be considered. When the suitable representations have been decided, one or more affinity measures (such as Hamming and Euclidean distance) are used to quantify the interaction elements of the systems. Finally, the use of AIS algorithms or processes to govern the behaviour (dynamics) of the systems is needed.

• Algorithms: generation, negative selection, clonal selection, immune network, etc

• Affinity measures: the strength of a match between detector and intrusion signature.

• Shape space: the quantity and nature of the detector features.

• Application domain: Pattern processor technology applied to IP packets.


Level 1 & 2 Agent: Detector Creation & Learning Architecture

L2 Agent

OtherL2 Agent

OtherL2 Agent

1

2

3

4

5

6

7


General L1 detector life cycle: creation, false-positive testing, deployment efficacy or termination, mutation improvement, and long-term memory.

1. Candidate fuzzy detector semi-randomly created.2. Tolerization period tests immature candidates for false-positive matches.3. Mature & naïve candidates put into time limited service.4. Activated (B-cell) candidates wait for co-stimulation (by T-cells) to ensure “improvement” didn’t

produce auto-reactive result, non-activated & non-co-stimulated candidates die when time limit ends.5. Highest scoring co-stimulated candidates are remembered for

time-limited long term. 6. Co-stimulated candidates are cloned with structured mutations,

looking for improved (higher) activation scores. 7. Level 2 Agent insertion of activated candidates from other end-

points, and Level 2 Agent distribution of activated candidates to other end points.

Not necessary – new approach departs from a strict BIS model

Eventually it became evident that a cloning improvement GA process was not necessary, as a speculative fuzzy-detector match-event could simply snatch and record the signature that caused the event. The realization: BIS works with complimentary patterns rather than duplicate patterns, and can only create through trial and error a complimentary pattern with best fit.This departure from the BIS model was initially uncomfortable, threw into doubt our high fidelity objective. We got over it. Coverage of pattern space (detector quantity and diversity) is the high fidelity issue, not process mirroring.


Detection Categories (Types)

Rather than categorize by attack type (like UTMs do), of which there is a never-ending list as new types emerge, our categories relate to the detection philosophy:

Automated learning of spatial and temporal pattern features within a fixed set of generic pattern structures.

A syn flood attack, e.g. is one instance within the temporal-connection pattern category.

• Spatial means a collection of features at one instant (packet/log-entry) in time that is detected as a pattern-of-interest.

• Temporal means multi-packet/multi-log-entry features that are detected as a pattern.• Correlative (unaddressed here) means multi-flow/multi-log features detected as a pattern.

Spatial Connection

Spatial Content

Temporal Connection

Temporal Content

L1 packet headers (feasibility demo for TCP/UDP/ICMP).

L1 deep packet inspection (feasibility demo for HTTP/SMTP???).

L2 multi-packet header detectors (feasibility demo for TCP/UDP/ICMP).

L2 multi-packet content detectors (feasibility demo for HTTP/SMTP???).


Phase 1 Initial FocusIPv4 packet-header detection – single packet-header signature patterns (spatial connection category)

Three elements to a pattern signature: address – port – type • Address: 4 bytes - Only the non-host address is of interest.• Port: 2 bytes - Only the destination port is of interest.• Types: 3 bits covers 8 types – (TCP, UDP, ICMP, other) x (incoming, outgoing)

The L1-Agent preprocessor/controller selects relevant features from network packets and feeds them as condensed “feature packets” to the pattern processor

L1 Agent• conventional processor/memory• detector generator• feature packet assembly• pattern processor controller

Pattern Processor• special purpose chip• detectors in nursery• detectors in service• detectors in memory

networkpackets

IPv4feature packets

detectionalert

L2 Agent


Feature Cells and Finite State Machines(Illustrative example of pattern processor capability)

256-bit associative memorymulti-feature detectors (MFD).

All active MFDs are indexed by the input stream’s current byte value.

If the index finds a set bit, the next MFD is activated and looks at the next stream byte, else the process dies.

IPv4 address port type

7 multi-feature detectors “connected”as a finite state machine (FSM)

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○○

start end

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○○


IPv4 address192.168.1.44

type2

○○

≈ ○○

○○

○○

○○

○○

○○

○●

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○●

○○

≈ ○○

○○

○○

○○

●○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

●○

○○

≈ ○○

○○

○○

○○

○●

○○

○○

○○

○○

○○

≈ ○○

○○

●○

○○

○○

○○

○○

○○

○○

start end

192.168.1.44, 0.118, 2

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○●

○○

Loaded with 7 values

port0.118








○○

≈ ○○

○○

○○

○○

○○

○○

○●

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○●

○○

≈ ○○

○○

○○

○○

●○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

●○

○○

≈ ○○

○○

○○

○○

○●

○○

○○

○○

○○

○○

≈ ○○

○○

●○

○○

○○

○○

○○

○○

○○

start end

192.168.1.44, 0.118, 2

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○●

○○

Processing Data Stream








○○

≈ ○○

○○

○○

○○

○○

○○

○●

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

○●

○○

≈ ○○

○○

○○

○○

●○

○○

○○

○○

○○

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○○

●○

○○

≈ ○○

○○

○○

○○

○●

○○

○○

○○

○○

○○

≈ ○○

○○

●○

○○

○○

○○

○○

○○

○○

start end

192.168.1.44, 0.118, 2

○○

≈ ○○

○○

○○

○○

○○

○○

○○

○●

○○







Very Large Scale Anomaly DetectorPatent Pending


Fundamental Elements

12 98 126 25 41 250 7 16 13 123 255 0 0 98 1

feature packet feature value

feature stream

patternspace

255 255 255 255 255 255

255 255 255 255 255 254

255 255 255 255 255 253

255 255 255 255 255 0

255 255 255 255 254 255

255 255 255 255 254 254

255 255 255 255 254 253

255 255 255 255 254 0

0 0 0 0 0 0

98 126 25 41 250 7

98 126 25 41 11 0

16 13 123 255 0 0

patternlist

pattern

pattern space contains2566 = 2.81x1014

unique patterns

for patterns consistingof 6 feature valueswith range 0-255


11 ≈ 001010110100101010

00 ≈ 001111110010101011

10 ≈ 001100110110111100

11 ≈ 110011100011101111

01 ≈ 011111110110101001

Gang Detector (GD)

10 ≈ 001001110010101111

Gang Detector(eg, with seven multi-feature detectors)

Multi-Feature Detectors(MFD, variable size)

Feature Indicator(1-bit)

Non-Feature Indicator(0-bit)

10101111

Pattern (or Pattern Path)


11 ≈ 001010110100101010

00 ≈ 001111110010101011

10 ≈ 001100110110111100

11 ≈ 110011100011101111

01 ≈ 011111110110101001

Gang Detector (GD)

10 ≈ 001001110010101111

10101111

A GD is implemented as a 2 dimensional bit array, with each column corresponding

to an MFD of independent size, but typically a max of 256 to accommodate associative addressing (indexing) by an

8-bit Feature Packet byte.

A Feature Indicator is a 1-bit in any or all of the possible index values.

One GD with all Feature Indicators present (all bits set to 1) would have

256x256x256x256x256x256x8 =

2.6x1015 unique Pattern Paths.

This many unique patterns would be represented in just (6x32)+1 =

193 8-bit data bytes.

If each of these patterns were in a pattern list, seven times the number of possible

patterns in data bytes would be required,

~1016 data bytes in contrast. Gang Detector(eg, with seven multi-feature detectors)






11 ≈ 001010110100101010

00 ≈ 001111110010101011

10 ≈ 001100110110111100

11 ≈ 110011100011101111

01 ≈ 011111110110101001

Gang Detector (GD)

10 ≈ 001001110010101111

10101111

A GD is implemented as a 2 dimensional bit array, with each column corresponding

to an MFD of independent size, but typically a max of 256 to accommodate associative addressing (indexing) by an

8-bit Feature Packet byte.

A Feature Indicator is a 1-bit in any or all of the possible index values.

One GD with all Feature Indicators present (all bits set to 1) would have

256x256x256x256x256x256x8 =

2.6x1015 unique Pattern Paths.

This many unique patterns would be represented in just (6x32)+1 =

193 8-bit data bytes.

If each of these patterns were in a pattern list, seven times the number of possible

patterns in data bytes would be required,

~1016 data bytes in contrast. Gang Detector(eg, with seven multi-feature detectors)





a unique benefit of the approach


00 ≈ 000000000000001000

00 ≈ 000000000000100000

00 ≈ 000000000000100000

00 ≈ 000001000000000000

00 ≈ 000000000100000000

00 ≈ 001000000000000000

00001000

7 Feature Indicators = 1 Pattern (Path)

Gang Detector (GD)


00 ≈ 000000000000001000

00 ≈ 000000000000100000

00 ≈ 000000000000100000

00 ≈ 000001000000000000

00 ≈ 010000000100000000

00 ≈ 001000000000000000

00001000

7 Feature Indicators = 1 Pattern (Path) 8 Feature Indicators = 2 Patterns (Paths)

Gang Detector (GD)


00 ≈ 000000000000001000

00 ≈ 000000000000100000

00 ≈ 000000100000100000

00 ≈ 000001000000000000

00 ≈ 010000000100000000

00 ≈ 001000000000000000

00001000

7 Feature Indicators = 1 Pattern (Path) 8 Feature Indicators = 2 Patterns (Paths) 9 Feature Indicators = 4 Patterns (Paths)

Gang Detector (GD)


00 ≈ 000000000100001000

00 ≈ 000000000000100000

00 ≈ 000000100000100000

00 ≈ 000001000000000000

00 ≈ 010000000100000000

00 ≈ 001000000000000000

00001000

Adding a single Feature Indicator increases the Patterns (Paths) by a factor of 2, an exponential increase.

An application might create a new GD with the same percentage of Feature Indicators in every MFD.

If that were 50%, with six MFDs of size 256 and one of size 8, the total number of Patterns (Paths) upon creation would be

128x128x128x128x128x128x4=

1.8x1013 patternsDetectable at data stream feed speed

independent of the number of patterns 7 Feature Indicators = 1 Pattern (Path) 8 Feature Indicators = 2 Patterns (Paths) 9 Feature Indicators = 4 Patterns (Paths)10 Feature Indicators = 8 Patterns (Paths)

Gang Detector (GD)


00 ≈ 000000000100001000

00 ≈ 000000000000100000

00 ≈ 000000100000100000

00 ≈ 000001000000000000

00 ≈ 010000000100000000

00 ≈ 001000000000000000

00001000

Adding a single Feature Indicator increases the Patterns (Paths) by a factor of 2, an exponential increase.

An application might create a new GD with the same percentage of Feature Indicators in every MFD.

If that were 50%, with six MFDs of size 256 and one of size 8, the total number of Patterns (Paths) upon creation would be

128x128x128x128x128x128x4=

1.8x1013 patternsDetectable at data-stream feed-speed

independent of the number of patterns 7 Feature Indicators = 1 Pattern (Path) 8 Feature Indicators = 2 Patterns (Paths) 9 Feature Indicators = 4 Patterns (Paths)10 Feature Indicators = 8 Patterns (Paths)

Gang Detector (GD)

a unique benefit of the approach


Gang Detector Function and Operation

Create a newGang Detector (GD)

Mature new GDin the Nursery

Use GDs todetect anomalies

Insert mature GDinto Service

Remove GDsfrom Service

GDCreation

GDMaturation

GDInsertion

GDDetection

GDRemoval

GDN1 GDN2 GDNn GDS1 GDS2 GDSm

Nursery Set Service Set

DM1 DM2 DMm

Memory Set

(single pattern detectors, not GDs)



50% of Feature Indicators set across 7 MFDs (6@256 & 1@8)

Multiple gang detectors covering slightly-overlapping portions of total pattern space collectively increase the total coverage of pattern space.

Create a newGang Detector (GD)

Mature new GDin the Nursery

Use GDs todetect anomalies

Insert mature GDinto Service

Remove GDsfrom Service

GDCreation

GDMaturation

GDInsertion

GDDetection

GDRemoval

GDN1 GDN2 GDNn GDS1 GDS2 GDSm

Nursery Set Service Set

DM1 DM2 DMm

Memory Set

(single pattern detectors, not GDs)

99.97% coveragewith 512 GDs



Maturation and detection happen in parallel, processing the same Feature Packet Stream. Thus a single Feature Packet Stream is used for maturing new GDs in the Nursery Set and detecting anomalies in the Service Set, simultaneously.

GD Detection

For everyFeature Value 16 in Feature Packet…

Start at next Feature Packet in

a Feature Stream

Indicate that a Pattern Path match

has occurred

FeaturePacket

finished?

Docorresponding

Multi-Feature Detectorscontain a Feature Indicator

at the location indexedby the Feature

Value?

NO

YES

YES

NO

GD Maturation

For everyFeature Value in a Feature Packet…

Start at next Feature Packet in

a Feature Stream

Choose oneMFD in GD

Remove corresponding

Feature Indicator

FeaturePacket

finished?

Docorresponding

Multi-Feature Detectorscontain a Feature Indicator

at the location indexedby the Feature

Value 16?

NO

YES

YES

NO

done


IPv4 Pattern-Space Coverage: c=6

1 29 57 85 1131411691972252532813093373653934214494775050.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Pattern-Space Coverage by Number of GDs

64 128 192 256 320 384 448 5120.00%

10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%

100.00%

63.50%

86.68%95.14% 98.23% 99.35% 99.76% 99.91% 99.97%

% Coverage by number of GDs

1 29 57 85 1131411691972252532813093373653934214494775050.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Pattern-Space Coverage by Number of GDs

64 128 192 256 320 384 448 5120.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

31.29%

52.79%

67.56%

77.71%84.69%

89.48% 92.77% 95.03%

% Coverage by number of GDs

IPv6 Pattern-Space Coverage: c=32

50% cardinality

85% cardinality


Coverage as Function of Cardinality

Cardinality losses justify the value of refresh-cycling the in-service GDs, and sharing results with other endpoint agents

accelerating decline in coverageas cardinality drops,

40% thought comfortable threshold


6 MFDs at 40% = 98.35% at 1024 GDsCoverage of 32 MFDs Declines Fast


Learning Curve Comparing Two Training Methods

range expanded 6x


Very Large Scale Anomaly Detector – Other App DomainsThe SORNS example is a specific domain instance of the more general “normal vs. anomalous behavior” classification problem. Of interest elsewhere, for instance: • monitoring the operational behaviors of a swarm of unmanned autonomous

weapons sent on a war fighting mission, • video image monitoring of human behavior in terrorist target areas like airports, • insider threat behavior,• monitoring machine and process behaviors in factories (anti-Stuxnet),• monitoring critical infrastructure operating behaviors, say for financial-market

transactions.

Perhaps most interesting, the human brain appears to work on anomaly detection• it appears to select and store (learn) pattern features for uniqueness and rarity• children learn with attention to things that are different and new• it doesn’t learn quickly, but does recognize patterns “immediately”• it can learn both by download (taught) and by speculative discovery


JTRS Network TestingThe testing for sufficient device security can never end, because the attacks on security evolve rapidly with intent to breech the device in new effective ways.Something “transparent” is needed on board every device that can monitor its behavior continuously for abnormalities, signal that something is amiss, and provide mediation options and field-operational data.

Transparency is key in multiple dimensions:• Does not require program intervention (e.g., no action required by anybody except test personnel) • Does not interfere with or degrade device functional performance.• Does not rely on external support services (e.g., McAfee daily updates).• Does not require external control or collaboration (e.g., black box self sufficient).

Physical Configuration PhasesP1: sniff the host traffic with a (centralized) stand alone wireless device located anywhere in wireless

range. P2: sniff host traffic on-board the host (physically attached).P3: integrated with host architecture and can take direct mediation action.

NetworkInterface

SystemFunctions

End-PointMonitor

Radio/Device

P3

NetworkInterface

SystemFunctions

Radio/Device

P2

End-PointMonitor

End-PointMonitor

End-PointMonitor

End-PointMonitor

NetworkInterface

Test Central

P1


The InZero® Security Platform provides a new approach to protecting PCs from malware. Rather than scanning suspect files with a virus scanner that looks for an ever-increasing number of malware signatures, the Gateway’s security architecture is based on a hardware sandbox that can safely execute malicious applications without damaging the Gateway or the PC. The hardware sandbox is shown in red because it permits the sandbox applications (e.g., browser) to safely execute arbitrarily dangerous application data files from the internet and from the PC. However, the design of the sandbox prevents any malware in these data files from modifying the Gateway’s software or writing any data used by the operating system. Physical separation prevents the malware from infecting your PC.

A Bump on the Network Wire…also wireless and USB filter

One possible implementation style:


Very Large Scale Anomaly Detector - SummaryGang Detector (seven-feature pattern example)• Memory breakthrough: 193 bytes vs. 1016 bytes for pattern storage• Coverage breakthrough: 512 GDs covers 99.97% of pattern space at 1 endpoint• Good coverage does not require high coverage at any endpoint:

• Endpoint multiples boost total coverage with same coverage curve• Endpoint refresh boosts total coverage with same coverage curve

• Custom anomaly detection – no two installations alike (in pattern content) • Dynamically adaptable to local situation

Pattern Processor without tradeoffs• Speed breakthrough: speed independent of number/size of patterns • Capacity breakthrough: affordable massive pattern counts

Potential Product Package• Transparent to endpoint: no latency or throttling of comm speed • Transparent to network: no accommodations required• Transparent to budget: no signature subscription• Transparent to installer: bump on the wire installation


A Fairer Fight




SecurityAgent (SA)


SecurityCommunities

Self Organizing SystemsDynamic Attack

Dynamic attack includes human and systemic adaptive control – unable to know the detection detail and capability.

Anomaly Sensors collaborate within a network, dynamically adjusting to local situations, and can also collaborate among networks.

AD AD

AA AA

SD SD

SA SA

Level-2Agent (L2)

SORNS Net Domain

SORNSCommunities L3 L3

L2 L2

Just one self-organizing security exampleof many more to come


Other Aware-Systems Domainsapplications in non-cyber domains

sensors are proliferating – sensemaking needs attention


Systems of Systems – Always There, Now Aware26Jan2011, Ford Previews Vehicle-To-Vehicle Tech At Washington Auto Show

A dedicated short-range WiFi system on a secure channel one-ups radar safety systems by allowing full 360-degree coverage even when there's no direct line of sight. Vehicle-to-vehicle warning systems could address nearly 80 percent of reported crashes not involving drunk drivers.

Prototypes to tour the U.S. this spring.

V2V tech could move to handheld devices, bringing the capability to all cars


Automated Disease Surveillance

Joseph Lombardo, Sheri Lewis, Rich Wojcik, and Wayne Loschen. 2010. Systems Engineering to Support Discovery of Threats to Public Health. INSIGHT 13(4), December.

Requirements Challenges

The requirements include a significant reduction in the time for the recognition of a significant health event so that an effective public-health response can be launched to minimize casualties.

The system must be able to recognize emerging health risks for both naturally occurring infectious diseases and those that are intentional. False positives must be kept to a minimum, and periodic evaluation is needed to ensure that the system provides the performance needed.

For an automated disease-surveillance system to achieve timely positive detection of a covert attack of weaponized bacillus anthracis spores, the system must rely on the behaviors of individuals manifesting early symptoms of the disease. The system may also need to rely on other sources of information not used in a clinical setting.

Electronic Surveillance System for the Early Notification of Community-based Epidemics

ESSENCE (Lombardo and Buckeridge 2007)


Smart Roads. Smart Bridges.

7Feb2009, WSJ, http://online.wsj.com/article/SB123447510631779255.html

SMART MOVES Freeway signs give estimated travel times and other information on Interstate Highway 80 in the San Francisco Bay area.Radio receivers are installed along several freeways in the San Francisco Bay area that read the electronic toll tags in passing cars. One promising avenue: real-time information about road conditions, traffic jams and other events. The next generation of technologies promises to get that news -- and even more detailed information -- directly to drivers in their cars.

Gi-Lu Cable-Stay Bridge in Taiwan

has wireless sensors and

accelerometers that monitor

its structural health.

www.scientificamerican.com/slideshow.cfm?id=smart-bridges-harness-tech&photo_id=EF0F0341-A452-806C-8CDD679765CAA94B© C.H. LOH, NATIONAL TAIWAN UNIVERSITY AND JEROME LYNCH, UNIVERSITY OF MICHIGAN AT ANN ARBOR

http://online.wsj.com/article/SB123447510631779255.html


Securing the Smart Grid:Next Generation Power Grid Security

Smart grids are a reality and the future, and they promise greater reliability, affordability, efficiency and, hopefully, a better and environmentally cleaner exploitation of available resources. But all that brings to light new threats to the grids. What does that entail and what can we do to defend them - these are the two main questions that this book offers the answers to.

You'll find out more about the threats to smart grids: natural, individual and organizational threats. A special part of the chapter is dedicated to the hacker threat and its various incarnations and motives. The impacts of these threats on utility companies and others are next, with various believable scenarios that point out the threats, their attack vectors and their impacts. From threats to individuals to those to entire countries - it makes you realize what the danger really is.Review: Zeljka Zorz, 6Dec2010, www.net-security.org/review.php?id=240

Chapter 2 provides an overview of the threats and impacts of smart metering at the consumer level. With the benefits come security and privacy issues. Leaving security to vendors of home-based products has traditionally not been met with much success. Chapter 4 notes … An important group working on this is the NIST Cyber Security Working Group (CSWG). The primary goal of the CSWG is to develop an overall cyber security strategy for the smart grid. This strategy addresses prevention, detection, response, and recovery. The CSWG recently created NISTIR 7628 — Guidelines for Smart Grid Cyber Security. Review: samzenpus, 5Jan2010, http://books.slashdot.org/story/11/01/05/1251224/Securing-the-Smart-Grid?from=headlines

Fractal at national transmission and local distribution levels.

http://www.google.com/url?q=http://collaborate.nist.gov/twiki-sggrid/bin/view/SmartGrid/CyberSecurityCTG&sa=D&sntz=1&usg=AFQjCNFo6yOW-nlSDomGS0xqt41-7Fe7SA






http://www.google.com/url?q=http://csrc.nist.gov/publications/PubsNISTIRs.html#NIST-IR-7628&sa=D&sntz=1&usg=AFQjCNHJZxh_RN7cEF-Bm-tEQn3NLpswtw





Physical Security & Smart BuildingsSensors on the property, at the building entry ways, within the building.IR motion, video, sound, pressure, ….ID entry ways (key card, fingerprint, iris, facial recognition…)RFID individual-human tracking within a building


Assessing Fleet Characteristics Useful for SensingMichael J. Ravnitzky, Offering Sensor Network Services Using the Postal Delivery Vehicle Fleet, 18th Conference on Postal and Delivery Economics,

Center for Research in Regulated Industries, Porvoo, Finland, June 2-5, 2010http://www.prc.gov/(S(je2vev45qxfhtai2g3npcczd))/prc-docs/newsroom/techpapers/Ravnitzky Postal Sensors Paper 070910-MJR-1_1191.pdf

Fleet Type

Single National Owner

Regular Routes

Time on the Road

Universal Geographic Coverage

Centralized Maintenanc

e

Geographic Flexibility/ Selectivity

Taxis X

Police Cars X X

City Buses X X X

School Buses X X

City Fleet X

UPS/FedEx X Limited X X X Limited

Postal Trucks X X X X X X

From Slashdot: The US Postal Service may face insolvency by 2011 (it lost $8.5 billion last year). An op-ed piece in the December 17, 2010 New York Times proposed an interesting business idea for the Postal Service: use postal trucks as a giant fleet of mobile sensor platforms.Think Google Streetview on steroids. The trucks could be outfitted with a variety of sensors (security, environmental, RF ...)

http://www.nytimes.com/glogin?URI=http://www.nytimes.com/2010/12/18/opinion/18ravnitzky.html&OQ=_rQ3D4&OP=5e2eaf78Q2F)-DE)UQ2219!Q22Q22_Q51)Q51IQ26I)Q26Q51)Q26Q2F)Q22xQ7B(Q7BQ22()Q26Q2F!pQ3A(Q7B_WNQ60Q2B,_.l

http://www.nytimes.com/glogin?URI=http://www.nytimes.com/2010/12/18/opinion/18ravnitzky.html&OQ=_rQ3D4&OP=5e2eaf78Q2F)-DE)UQ2219!Q22Q22_Q51)Q51IQ26I)Q26Q51)Q26Q2F)Q22xQ7B(Q7BQ22()Q26Q2F!pQ3A(Q7B_WNQ60Q2B,_.l


Potential Applications for Postal Truck-Borne Mobile SensorsMichael J. Ravnitzky, Offering Sensor Network Services Using the Postal Delivery Vehicle Fleet, 18th Conference on Postal and Delivery Economics,

Center for Research in Regulated Industries, Porvoo, Finland, June 2-5, 2010http://www.prc.gov/(S(je2vev45qxfhtai2g3npcczd))/prc-docs/newsroom/techpapers/Ravnitzky Postal Sensors Paper 070910-MJR-1_1191.pdf

Application Description Likely Customer BaseChemical Agents DHS, StatesBiological Agents DHS, StatesRadiological Materials DOE, DHS, StatesAir Quality EPA, States, CitiesEnvironmental Sensing EPA, States, USDA, CitiesRadio/Television Signal Strength FCC, TelecomsWireless Signal Strength FCC, TelecomsWeather/ Meteorological National Weather ServicePothole Mapping/Road Assessment Public Works DepartmentsNatural Gas Leaks Gas UtilitiesLicense Plate Scanning Law EnforcementMethamphetamine Labs Law EnforcementMarijuana Farms/ Drug Depots Law EnforcementIllicit Explosives Production Law EnforcementPhoto Imaging Google, Law Enforcement, Local GovernmentsNoise Profiling/ Acoustic Signature Zoning, Cities, ResearchPest Control State, County GovernmentsBiological Surveys Scientific CommunityNuclear Radiation Leaks NRC, UtilitiesElectric Field Mapping EPA, Cities, Scientific CommunityMagnetic Field Mapping EPA, Cities, Scientific CommunityOther Scientific Investigation DoD, DOE, Scientific Community,Meter Reading Universities


Personal weather station is alien chic "Weather information from thousands of personal weather stations are being used for weather forecasting by several private and government agencies, including the National Oceanic and Atmospheric Administration (NOAA) and the Dept. of Homeland Security (DHS). The Citizens Weather Observation Program (CWOP) was created by a few amateur radio operators experimenting with transmitting weather data with packet radios, but it has expanded to include Internet-only weather stations as well. As of September 2007, nearly 5,000 stations worldwide reported weather data regularly to CWOP's FindU database.In Feb 2007 (http://www.pnl.gov/main/publications/external/technical_reports/PNNL-16422.pdf) DHS listed CWOP as a national asset to the 'BioWatch' Network, stating that data from personal weather stations could be useful in weather forecasts for hazardous releases. In 2007, the FindU server received 422,262,687 weather reports which is a 29.5% increase over 2006. [http://science.slashdot.org/article.pl?sid=08/01/19/1835237]

No longer do home forecasting gadgets look like hospital equipment thanks to Oregon Scientific's efforts to add an aesthetic dimension to its products. Its latest offering looks more like a retro sci-fi movie prop than something used to guess whether you should bring a sweater to the picnic.

[http://crave.cnet.com/8301-1_105-9842340-1.html]


Pothole Sensors in Every Carwww.boston.com/news/local/massachusetts/articles/2011/02/09/weapons_in_the_battle_vs_potholes/

9Feb2011: The City of Boston is releasing an app called Street Bump that uses the accelerometer in your smartphone to automatically report bumps in the road as you drive over them. The application relies on two components embedded in iPhones, Android phones, and many other mobile devices: the accelerometer and the Global Positioning System receiver. The accelerometer, which determines the direction and acceleration of a phone’s movement, can be harnessed to identify when a phone resting on a dashboard or in a cupholder in a moving car has hit a bump; the GPS receiver can determine by satellite just where that bump is located“We’re constantly looking for new ways to make sure that roads are as smooth as they possibly can be, and we believe that Street Bump is a first-in-the nation app,’’ said Chris Osgood, one half of the two-man Urban Mechanics office.Osgood and Nigel Jacob, New Urban Mechanics cochairman, have developed the prototype with Fabio Carrera, a professor at Worcester Polytechnic Institute, and Joshua Thorp and Stephen Guerin, developers from the Santa Fe Complex, a civic-minded technology and design think tank in New Mexico.“It’s a new kind of volunteerism,’’ Jacob said. “It’s not volunteering your sweat equity. It’s volunteering the devices that are in your pocket to help the city.’’

[email protected], 70 IBM Tivoli

[email protected], 1 Toward Systems With a Will to Live SORNS Self Organizing Resilient Network...

Documents

Transcript of [email protected], 1 Toward Systems With a Will to Live SORNS Self Organizing Resilient Network...