1

Understanding Cyber Warfare: Broadening and Scaling up IBL Models

Collaborative Project: Coty Gonzalez (CMU) & Nancy Cooke (ASU)

Noam Ben-Asher, Ph.D. Post-Doctoral Fellow – CMU

Prashanth Rajivan

Graduate Student - ASU

Broadening and Scaling up IBL models

2

Modeling detection with Instance-Based Learning Theory (Dutt, Ahn, Gonzalez, 2011, 2012)

Defender

Defender Attacker

From Individual Decisions from Experience to Behavioral Game Theory: Lessons for Cyber Security (Gonzalez, 2013) Perspectives from Cognitive Engineering on Cyber Security. (Cooke et al., 2012).

Individual (Defender). Cognitive theories, Memory and individual behavior

Interdependencies (Defender and Attacker) Behavioral Game Theory

Interdependencies and Group Dynamics (Defender and Attacker within each individual) Behavioral Network Theory; Network science (& topology) Organizational Learning; Political and Social Science

Cyber Warfare: Attacker & Defender

The Cyber Warfare Simulation Environment and Multi-Agent Models (Ben-Asher, Rajivan, Cooke & Gonzalez, in preparation).

Instance-Based Learning Theory (IBLT) – Gonzalez, Lerch & Lebiere, 2003 We evolve to select actions that have led to best outcomes in similar situations

(contingencies) in the past.

The Individual: Cognitive Theories, Cognitive Architectures

3

IBLT - Formalizes invariant cognitive representations and processes and provides theoretical boundaries of human cognition 1. Each Situation-Decision-Utility (SDU)

combination is created as an instance in memory when the outcome is experienced

2. Each instance has an “activation” and a “retrieval probability” (Based on ACT-R memory mechanisms)

3. A Utility for each instance is calculated (by combining retrieval probability and value)

4. The option with the highest utility is selected

• SDU instance: • Activation: simplification of ACT-R’s mechanism:

• Probability of retrieval: a function of memory Activation:

• Choose the decision with the highest utility (“blended” value):

The IBL essential decision mechanisms

4

Attributes Decision Outcome Own

Power Own Asset

Opponent Power

Opponent Asset

Attack/No Attack

Amount of Assets gained\Lost

Cyber security is not a “game against nature”

5

• Many (most/all) relationships between two entities can be characterized using Game Theory formulations

• The interdependencies between two entities have been successfully modeled using two instantiations of IBL agents in traditional Game Theory formulations (Prisoner’s Dilemma, Chicken Game)

• The IBL model may represent a Defender and an Attacker

1. Game theory prescribes solutions (equilibria) that are often not in accordance with actual observed human behavior: – Human cognitive limitations, learning, memory, adaptation

2. Traditional game theory often assume full information and ignore partial, asymmetric and gradual discovery of information – Information ladder as cognitive factors

3. Traditional game theory often ignore other social variables (e.g. Power; reciprocity; trust) – Integrate social effects in cognitive process

4. Traditional game theory problems are formulated in terms of two individual decision makers that ignore group dynamics – Scaling from inter-personal to inter/intra group dynamics emerging

from cognitive agent interactions

Some challenges in using BGT in Cyber Security:

6

• Cyber attacks against the users, servers, and infrastructure are a reality

• Each entity may be an attacker and a defender

• Involving countries, groups, creating coalitions

• Inter and Intra-group conflict dynamics

Cyber security is beyond a conflict between an “Attacker and Defender”: Cyber Warfare

7

• N players – each is represented by a cognitive agent/model that makes decisions and learn from their outcomes.

– Whether or not to attack any of the other agents.

• Agents are countries, organizations, facilities, etc.

• Attributes of the agent represent real world characteristics like:

– Power of cyber security infrastructure and vulnerability.

– Value of assets an agent has.

• The incentive to initiate an attack is to get more assets.

• But, there are risks and costs involved in attacking other agents.

The Cyber Warfare Game

8

Demo

9

Model Overview

10

Phase 1: Scan

participating agents

Find the most attractive agent

to attack

Update cost and rewards Phase 2: Update

memory based on the outcome

Phase 2: Make a decision

Attack or Not Attack

Pre-Phase: Create 9 types of Agents

Phase 1: Find an Opponent

11

Scan participating

agents

Calculate utility value

for each agent

Choose most attractive agent to attack

Phase 2: Make a decision

12

Activate memory for the chosen opponent

Calculate Utility value (Attack and Not Attack)

Highest Utility?

Update cost and rewards Continue..

Phase 2: Update

memory based on the

outcome

ATTACK

Don’t Attack

Won?

Payoff: Attack Cost

Payoff: Rewards

- Cost

Lost

Won

CyberWar Model Interface

13

• Agents are defined by the combination of their asset and power, i.e., AssetPower.

– High Power: Red – Medium Power: Green – Low Power: Blue – Size: Amount of Assets

• Agents are not defined as

attackers or as defenders.

At each time tick and agent can launch only a single attack

14

Attacks at t Outcomes of the attacks at t

• 2 out of 6 low and medium power agents are suspended • High power agents dominate the network, all other agents have low

assets

Attacks have a direction

15

• A network with 9 different types agents – Power (High, Medium, Low) – Asset Value (High, Medium, Low)

• Each network was simulated for 1500 trials.

• 58 simulations with the same network setting.

• IBL Agents with d=5 and σ = 0.25

Simulating Cyber Warfare

16

• It takes about 100 ticks for the network size to stabilize on 6.5 agents.

• Then, the size of the network stays relatively stable for the rest of the simulation.

Results Network Size and Agents’ Downtime

17

Proportion of downtime according to Assets and Power

18

• Agents with power lower than 100 are suspended. • Low power agents are suspended more often compared to high

power agents.

Dynamics of Assets

High Power Agents Low Power Agents

High Asset Agents

Low Asset Agents

Aggression The Probability of Attacking Other Agents

High Power Agents Low Power Agents

High Asset Agents

Low Asset Agents

• Power is the main determinant of: – Loosing wars – getting suspended – Choosing attractive opponent behavior – Dynamics of assets over time

• Why are high power agents likely to attack other high power agents? – More available?

• Assets distribution: – Too high assets attracts attacks – Too low assets lead to high downtime

Summary of preliminary results

21

• Increase the action space of the agents with active (attack, defend) and passive (do-nothing) decisions.

• Allow an agent to attack several agents simultaneously. • Use cognitive and social attributes to generate different

types of agents and interactions • Examine the influence of the network:

– Size – Heterogeneity (distribution of power and assets) – Network topology

• Examine collaboration - coalitions of agents and distributed attacks.

Future Research (on Cyber War simulation only)

22

Slide Number 1Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Dynamics of AssetsAggression�The Probability of Attacking Other AgentsSlide Number 21Slide Number 22