United Nations Group of Governmental Experts Meeting:Lethal Autonomous Weapons Systems

Military Effects Panel

Dr. Lydia Kostopoulos@LKCYBER

November 14, 2017

College of Information and CyberspaceNational Defense University

U.S.A.

DISCLAIMER:

The views expressed in this presentation are those of the author and do not necessarily reflectthe views of the National Defense University, the College of Information and Cyberspace, or theUnited States Government.

TRUST CULTURE AVAILABILITY

Drivers for the Deployment of AI TechnologiesRational State Actors

TRUSTCULTURE AVAILABILITY

Autonomous machines perform as expected and within parameters outlined by human.

• Authenticity of the machine learning environment.

• Integrity of the data used.

Source: Geospatial World

HumanOnly

MachineOnlyHuman is

informed by machine.

Human is assisted by machine.

Human supervises work done by machine.

Close visual supervision

Remote supervision

Supervision through

alerts

With ability for human to modify, intervene and change or stop order/process.

Human gives command

without ability to change

course.

Human has confidence in

outcome.

Machine performs a specific, unambiguous task

Human gives a command in the form of objectives with guidance.

The machine has the the authority to execute full command without

reporting back.

Human has ability to

track machine’s activities.

Human has no ability to

track machine’s activities.

Machine performs a specific, unambiguous, or ambiguous task

Human-Machine Teaming Range for Lethal Autonomous Weapons Systems Discussion

Lydia Kostopoulos, PhD2017

Human gives a command in the form

of objectives with guidance.

The machine does not have the authority to execute full command

without reporting back.

Human has ability to track, modify,

intervene and/or stop the order or process.

Human has ability to modify,

intervene and/or stop the order or

process.

MachineOnlyHuman gives a command in the form of objectives with

guidance.

The machine has the the authority to execute full command without

reporting back.

Human has ability to

track machine’s activities.

Human has no ability to

track machine’s activities.

Machine performs a specific, unambiguous, or ambiguous task

Human gives a command in the form

of objectives with guidance.

The machine does not have the authority to execute full command

without reporting back.

Human has ability to track, modify,

intervene and/or stop the order or process.

Human has ability to modify,

intervene and/or stop the order or

process.

I. Reduction of cost (through process optimization, efficient management of limited resources)

II. Reducing burden on the soldier/commander (physically or intellectually)

Confidence Growth

III. Successful synthetic training & physical testing Demonstrated ability to successfully:- Operate on intent with minimal to no call-back for additional information or permissions.- Assimilate knowledge transfer from other autonomous systems’ data, experience, decisions etc (speed advantage).- Operate within “moral” bounds/limits assigned to the algorithm complying with rules of distinction and proportionality; while finding creative solutions to achieve mission objectives.- Communicate (to humans) actionable anticipatory intelligence in meaningful ways that directly fall inline with its mission and mandate; and to autonomously act successfully with the incorporation of anticipatory intelligence (speed advantage).

IV. Explainable AI/AlgorithmsAbility to explain (and confidently “fix”) AI algorithms will lead to greater trust in (expected) outcomes.

Detect, Select, Engage

TRUSTCULTURE

AVAILABILITY

“New” technology is normal

Anecdote

New technology is novel and interesting

New technology goes against

human nature

Digital Culture EvolutionFood for thought

• Acceptability of changing technology in battlefield, seeing technology as one with AI and a justifiable and “normal” and fair use of military tech assets to achieve mission success and political objectives.

• Moving forward, as digital natives become commanders, their perception of what types of weapons are acceptable in the battlefield may be different from ours.

• An AI 9/11 style attack conducted by a terrorist organization could push society to embrace the need to incorporate machine learning in all facets of society with a renewed drive to make LAWS fit within the values of our society.

TRUST CULTUREAVAILABILITY

• Availability refers to the availability of the technology to be used.

• The democratization of technologies also will reach hostile non-state political actors who do not abide by international norms.

• Tech won’t matter as much as creativity in it’s use that will bring decisive advantage.

CULTURE AVAILABILITY TRUST

Asymmetric Force Application: Alternative perspective for non-state hostile actors

[ex: terrorist organizations]

Culturally Acceptable

AchievableMilitary Effects

TechnologicallyPossible

LegallyPermissible

Area of probable lethal autonomous weapons usage

Culturally Acceptable

AchievableMilitary Effects

TechnologicallyPossible

Law Abiding Actors Non-Law Abiding Actors