International Technology Alliance inNetwork & Information Sciences

Dave Braines, David Mott, Simon Laws (IBM UK)Geeth de Mel, Tien Pham (ARL)

Dave Braines, David Mott, Simon Laws (IBM UK)Geeth de Mel, Tien Pham (ARL)

SPIE Defense Security & SensingSPIE Defense Security & SensingNext Generation AnalystNext Generation Analyst

Controlled EnglishControlled Englishto facilitate human/machine to facilitate human/machine

analytical processinganalytical processing

SPIE Defense Security & SensingSPIE Defense Security & SensingNext Generation AnalystNext Generation Analyst

Controlled EnglishControlled Englishto facilitate human/machine to facilitate human/machine

analytical processinganalytical processing

Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The USand UK Governments are authorized toreproduce and distribute reprints forGovernment purposes notwithstandingany copyright notation hereon.

Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The USand UK Governments are authorized toreproduce and distribute reprints forGovernment purposes notwithstandingany copyright notation hereon.

AgendaAgendaAgendaAgenda

Controlled English & the CE Store

Motivations & military relevance

Field Trial example

Other CE research areas

2

What is Controlled English?What is Controlled English?What is Controlled English?What is Controlled English?

3

A number of research collaborations, demonstrations and transition projects are experimenting with CE related capabilities both inside and outside the ITA programme

In our research activities CE has been the single core information representation format for humans and machines. No translation to underlying technical formats.

The Controlled English “ecosystem”: More than just a language representation Agent-based execution environment Multi-modal visualisation & interaction Focus on agility & flexibility

A Controlled Natural Language: Based on English, but formal and limited Directly processable by machine agents Can be used at design and/or run time Unifying information format:

• Model, facts, rules, queries, annotations & commands

• Rationale, hypotheses, assumptions & meta-data To support human/machine, machine/machine and

human/human interactions Fuse formal logic with social semantics

Natural

Language

Why are we researching this?Why are we researching this?Why are we researching this?Why are we researching this?

4ITA Peer Review, Sept. 2012The potential value of a “usable” semantic processing environment in the hands of non-

technical, domain-specialist users is very high, especially in dynamic situations.

Bringing capabilities of machinesand humans together

Human brain for insight & understanding

The key component is the human Harnessing “collective intelligence” Help make connections outside the system

Java

Controlled NaturalLanguage

XML

Logic

Prolog

• We would like thinking and processing to be as close as possible• We need a language that is both thinkable and processable. P

roce

ssin

g

Articulation as Language

Photographer: Sebastian Kaulitzki | Agency: Dreamstime.com

The CE StoreThe CE Store(aka the IBM Controlled Natural Language Processing Environment)(aka the IBM Controlled Natural Language Processing Environment)The CE StoreThe CE Store(aka the IBM Controlled Natural Language Processing Environment)(aka the IBM Controlled Natural Language Processing Environment)

5

Implemented in Java, default client runs in TomCat Core CE parsing engine:

Model, fact, query, rule, command, annotation, reified sentences Rationale integration, extensible meta-model

Default web-based user interface and client Customer in-memory Java based object persistence

Database persistence being considered for future version Customisable alerts and triggers APIs:

HTTP APIs for most common activities (JSON response) Internal Java programming APIs and extensible Agent development

environment Visual query building canvas (CE Query Builder or CEQB)

Available for download from: http://ibm.co/RDIa53 (IBM developerWorks)

Motivations and Military RelevanceMotivations and Military RelevanceMotivations and Military RelevanceMotivations and Military Relevance

6

Targeting coalition operations Dynamic and ad-hoc, but shared goals Opportunity to share information and insight

Multi-agent collaborative environment For human and machine agents

How can technology help? Rationale for explanation Facilitate different socio-cultural backgrounds

• e.g. moderate/observe information flow (to help misunderstanding) Improve operational tempo

• …to get inside the opponent decision cycle, or just be more effective Reduce communication overhead

• Don’t send information that can be re-inferred at the destination

The DCDP loop

Field Trial (LOSA)Field Trial (LOSA)Field Trial (LOSA)Field Trial (LOSA)

7

Late 2012, UKEvaluate sensors, systems, architectureHeterogeneous environmentBased around FOB activities

Situation awareness & agilityHard & soft fusion

This demo is driven ONLY by Controlled English

+ screenshots are generic – driven by CE

All material is unclassified

Pre-deployment activitiesPre-deployment activitiesPre-deployment activitiesPre-deployment activities

8

Geo-spatial preparation ahead of deployment:

Simple multi-modal integration

there is a building named b1 that has ’51.23’ as latitude and has ‘-1.74’ as longitude.

Dynamic model development

conceptualise a ~ building ~ B that is a spatial thing.

conceptualise a ~ ground feature ~ G that is a spatial thing.

Use of meta-model

the renderable concept 'building’ has '/icons/building.png' as icon file name.

Crowd-sourced Intelligence GatheringCrowd-sourced Intelligence GatheringCrowd-sourced Intelligence GatheringCrowd-sourced Intelligence Gathering

9

Imagery

Chat

Custom ‘PhotoProcessor’ Agent

there is a photo named 'IMG_0486.JPG' that has '/photos/IMG_0486.JPG' as image url and has '/photos/thumbnail.JPG' as thumbnail url and has '51.61500000' as latitude and has '-2.74616667' as longitude and has '52' as elevation and has '222.97' as bearing angle.

Standard JSON conversionthere is a field intelligence incident named 'inc_540' that has 'Droid810G' as device model and has '5164585396' as source id and has '51.614167' as latitude and has '-2.7475' as longitude and has '48' as elevation and has 'Local trader passing through with goods for market. Inspected and passed on' as body and has 'Alpha,US' as owner and has 'normal' as priority and has '2012-10-04 12:00:00.0' as timestamp and has 'INTEL' as type.

Need highly agile and responsive system

Dynamic / contextual filtering

Capture new insights

Anecdote: Linking informationAnecdote: Linking informationAnecdote: Linking informationAnecdote: Linking information

10

Observer: “Look, that photo is of that building”

New idea: Photos can be of thingsconceptualise the photo P ~ is of ~ the thing T.Now we can state:the photo ‘IMG_0486.JPG’ is of the building ‘b1’.

Observer: “Why can’t I see the building linked to the photo?”

Need to infer the inverse:conceptualise the thing T has the photo P as ~ imagery ~.Write a rule to infer:[thing has photo]if (the photo P is of the thing T) then (the thing T has the photo P as imagery).

Soft sources: Information extractionSoft sources: Information extractionSoft sources: Information extractionSoft sources: Information extraction

11

Other ITA research is looking at Information Extraction from natural language Not covered in detail in this paper Example shows detection of attributes for “fours small jeeps or cars”

Related to vehicles A sized group with cardinal size of “4”

Used in trial for: IED mentions Statements of

certainty Geo-location Associated

imagery

Hard sources: Agile integrationHard sources: Agile integrationHard sources: Agile integrationHard sources: Agile integration

12

Conversion of multiple “on the wire” formats: CSV, XML, JSON Used “Information Fabric” and external commercial / government sources Human led integration / mapping exercise

with machine led processing of events

Crude form of Situation Awareness

No integration with wider models / scenarios

No reasoning or inference …but lots of possibilities if

time allows

Good feedback from visitors and other trial participants

Possibility for visit in 2013

Other CE-related research areasOther CE-related research areasOther CE-related research areasOther CE-related research areas

On-going ITA research using Controlled English: Policy

A candidate light-weight, distributable, readable & executable policy language Trust & meta-data

CE reification to assign subjective logic trust values and propagate using inference and rationale

Collaborative coalition planningRich semantic model for collaborative coalition planning, specialising from general concepts of space and time, through problem solving to military planning

External interest: NATO, NS-CTA, TerraHarvest

Other SPIE papers: Context-rich semantic framework for effective D2D in coalition networks (8742-2, Monday) MIPS: A Service-Based Aid for Intelligence Analysis (8758-14, in this session) CE-SAM: a conversational interface for ISR mission support (8758-6, 17:20 today)

Email: dave_braines@uk.ibm.com or mottd@uk.ibm.com

Questions?Questions?

Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.

Main links: CE Store

http://ibm.co/RDIa53(from IBM developerWorks)

International Technology Alliancehttp://www.usukita.org

Controlled English resourceshttp://usukita.org/controlledEnglish