NCOIC Geospatial Interoperability Task Team Presentation

36
NCOIC Geospatial Interoperability Task Team Presentation Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ March 5, 2012 1 is not just where you put your data (cloud), but how you put it the

description

NCOIC Geospatial Interoperability Task Team Presentation . Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ March 5, 2012. - PowerPoint PPT Presentation

Transcript of NCOIC Geospatial Interoperability Task Team Presentation

Page 1: NCOIC Geospatial Interoperability Task Team Presentation

1

NCOIC Geospatial Interoperability Task Team Presentation

Dr. Brand NiemannDirector and Senior Enterprise Architect – Data Scientist

Semantic Communityhttp://semanticommunity.info/

AOL Government Bloggerhttp://gov.aol.com/bloggers/brand-niemann/

March 5, 2012

It is not just where you put your data (cloud), but how you put it there!

Page 2: NCOIC Geospatial Interoperability Task Team Presentation

2

Where I am Coming From?• I am a Data Scientist tasked with building semantically interoperable and

federated apps for:– Letitia Long, Director, National Geospatial-Intelligence Agency– Gus Hunt, CTO, Central Intelligence Agency– George Strawn, Director, National Coordination Office, OSTP, Executive Office of

the President• Staff of Aneesh Chopra, Former Federal CTO, Executive Office of the President

– Dennis Wisnosky, CTO, Department of Defense– Steve Cooper, CIO, FAA– Wyatt Kash, Editor in Chief, AOL Government– 13th SOA for eGov Conference, MITRE, McLean, VA, April 3, 2012

• I want to be a “go to person” like Brian Hurd was two years ago for the Intelligence Community and now is for a Fusion Center.– 2010 TUCON Slides.

Page 3: NCOIC Geospatial Interoperability Task Team Presentation

3

What Did I Hear Last Week• Keith Barber – Implementing NGIA On-line and On-demand

– GEOINT VISIONARY: Strengthening Analysis and Making Content More Accessible (see next slides)

• Boeing – Supply Chain Management for Disaster Relief• Deloitte – Healthcare for Disaster Relief• NATO – Geospatial Framework and Disaster Relief Use Case• NCOIC-MIT – Network Science• NCOIC-EADS – SOA Federation• Google Earth Builder – Cloud Service Broker to NGIA

– Geospatial solution that lets you publish your mapping data on our secure, cloud-based mapping platform and share it quickly and easily through Google Earth, Google Maps and Android Phones.

• Strata Conferences – Making Data Work– Data River, Data Finding Data, etc.

Page 4: NCOIC Geospatial Interoperability Task Team Presentation

4

What Did I Say Last Week?• I like the parallel track (conventional and semantic technologies) idea for

the pilot/demo, but just a data and apps layer would leave out unstructured data, if we do not get enough participation then will not have the “voice of industry” and multiple interoperable apps like the Federal SOA CoP does, and we at least need the global naming for all the resources and if we do we better make sure it supports agent technology (e.g. “data finding data”) like the Strata Conference videos describe.

• Just show me a demonstration of what Josh Lieberman is describing and I will be able to more fully respond. Does one of the OGC Testbeds show what he is describing?

• Maybe if we took the NATO Geospatial Framework and some NATO response data, we could jointly demo this. I will start working on this for next week’s meetings.

Page 5: NCOIC Geospatial Interoperability Task Team Presentation

5

GEOINT VISIONARY: Strengthening Analysis and Making Content More Accessible

https://www1.nga.mil/MediaRoom/LeadingStories/Pages/GIFArticle.aspx

Page 6: NCOIC Geospatial Interoperability Task Team Presentation

6

GEOINT VISIONARY: Strengthening Analysis and Making Content More Accessible

• Selected Excerpts:– Q: In your recent remarks to the GEOINT 2011 Symposium, you addressed what had

happened in the year since you outlined your vision of “Putting the Power of GEOINT in your hands.” How would you sum up your conclusions on what you have seen so far?• A: Taken together, the strategic initiatives will ensure our content is discoverable and accessible

through an open IT environment, allowing GEOINT users—inside and outside of NGA—to operate in a self-, assisted- and full-service environment and enabling analysts to do deeper and richer analysis.

– Q: Your address devoted a considerable amount of time to demonstrating some of the new mobile apps being developed by NGA. How would you characterize progress in that area?• A: This is an area that has a great deal of growth potential. We’ve had great initial success with

our application in support of the homeland security/domestic disaster support mission, sometimes building them on the fly in response to our analysts observing operations in the field. In addition, in support of our safety of navigation mission, we are delivering a mobile device-enabled set of air navigation aids, helping to streamline our processes, get updated information to the user faster and reduce paper product generation.

Page 7: NCOIC Geospatial Interoperability Task Team Presentation

7

GEOINT VISIONARY: Strengthening Analysis and Making Content More Accessible

• Selected Excerpts:– Q: Your 2010 vision also included a call for deepening analytic expertise. What do

you see as the key accomplishments and challenges in that area?• A: Building and deepening analytic expertise is a top priority. A key goal of the vision is to

provide an analytic environment where analysts spend their time doing analysis and not waste their time looking for data, or building spreadsheets, or signing onto multiple systems.

– Q: What do you see as NGA’s greatest challenges?• NGA’s traditional order-entry processes and supply chain model are outdated, and we are

working to adapt our role from a legacy producer of product to a broader role as a GEOINT service and application provider that enables end-users to configure GEOINT according to the needs of specific users, particularly in the expanding mission areas of homeland security and disaster relief.

– Q: What do you see as the agency’s most significant steps in the past year in using commercial imagery, and where do you see that going in the future?• These new products provide the U.S. government and our coalition partners a high quality

current image foundation built to Open Geospatial Consortium standards and available via web services.

Page 8: NCOIC Geospatial Interoperability Task Team Presentation

8

How Would I Build a New App?

• Goal: NATO Geospatial Framework and NATO Disaster Response

• Activities:– Check NGIA and NATO Web Sites for Sitemaps– Use Previous Work Like Global Disaster Knowledgebase– Build Google Search Index in Spreadsheets– Build Knowledgebases Based on the New Sitemap and

Schema Protocols– Federate the Distributed Knowledgebases for Analytics

Page 9: NCOIC Geospatial Interoperability Task Team Presentation

9

National Geospatial – Intelligence Agency

Site Map: None andNothing on NATO

http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration#Spotfire_Dashboard

Page 10: NCOIC Geospatial Interoperability Task Team Presentation

10

NATO

http://www.nato.int/cps/en/natolive/index.htm

Disaster Response

Page 11: NCOIC Geospatial Interoperability Task Team Presentation

11

NATO

• CIVIL EMERGENCY PLANNING:– Civil Emergency Planning (CEP) – Euro-Atlantic

Disaster Response Coordination Centre (EADRCC), The - – Civil Emergency Planning Committee (CEPC)

• Searches:• NGIA – No Results• Geospatial Framework:

– International geospatial symposium held at NATO HQ22 Apr. 2004 International geospatial symposium held at NATO HQ On 22 and 23 April 2004, NATO Headquarters hosted its first International Geospatial Symposium, which brought ...

http://www.nato.int/cps/en/SID-411BE335-F2A431FC/natolive/68147.htm

Page 12: NCOIC Geospatial Interoperability Task Team Presentation

12

International Geospatial Symposium Held at NATO Headquarters

• On 22 and 23 April 2004, NATO Headquarters hosted its first International Geospatial Symposium, which brought together geospatial experts from major international organisations.

• The symposium focused on the increasing importance of geospatial data - geographic data and imagery obtained from satellites - for the work of NATO and other international organisations.

• Detailed presentations were made by each invited geospatial team, providing a valuable insight into how they function on a day-to-day basis, their strengths and limitations, how geospatial data is used and the importance of their role within the organisations they serve.

• The symposium helped to forge new relationships and strengthen existing ones. It was also a first step in creating a framework for exchanging ideas and working methods.

• The participating international organisations were: the United Nations, the European Union and EU Satellite Centre, the World Food Programme, Eurocontrol, the European Environmental Agency, OSCE, and NATO.

http://www.nato.int/cps/en/natolive/news_20884.htm?selectedLocale=en

Page 13: NCOIC Geospatial Interoperability Task Team Presentation

13

The Euro-Atlantic Disaster Response Coordination Centre

The EADRCC’s main function is to coordinate the response of NATO and partner countries to natural or man-made disasters within the Euro-Atlantic area. The Centre has guided consequence management efforts in more than fourty-five emergencies, including fighting floods and forest fires and dealing with the aftermath of earthquakes.

http://www.nato.int/cps/en/natolive/topics_52057.htm?

Build Knowledgebases!

Page 14: NCOIC Geospatial Interoperability Task Team Presentation

14

The Euro-Atlantic Disaster Response Coordination Centre

http://www.nato.int/eadrcc/

Page 15: NCOIC Geospatial Interoperability Task Team Presentation

15

Google Search:NATO Geospatial Framework

Paid Advertisement!

ESRI 2011 PDF File!

My Slides for January 20, 2012!

NATO BRITE Map Server Portal!Web Addresses From Robert Kruse:http://www.nato.int/cps/en/natolive/topics.htm?query=E https://www.cimicweb.org http://www.gdacs.com/ http://vosocc.unocha.org/ http://www.reliefweb.int http://www.ibm.com/connect/ibm/us/en/resources/emenaker/ http://www.nato.int/cps/en/natolive/index.htm http://www.simulationteam.com

NATO Geospatial Orientation Course

OGC Helps NATO

Defence Geospatial Intelligence

FedBizOps 2006

Page 16: NCOIC Geospatial Interoperability Task Team Presentation

16

The NATO Core Geographic Services System

http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration/NATO_Disaster_Response#The_NATO_Core_Geographic_Services_System

Contact Information:John F. Teufert, NC3A Geo-OfficerCapability Area Team 6, NC3AOude Waalsdorperweg 61, 2597 AK, The Hague, Netherlands

Page 17: NCOIC Geospatial Interoperability Task Team Presentation

17

NATO BRITE Map Server Portal

https://transnet.act.nato.int/WISE/BRITE/BRITEGraph/BRITEOverv/index_html

SOA!

Site Map

Page 18: NCOIC Geospatial Interoperability Task Team Presentation

18

NATO ACTTransformation Network Portal

https://transnet.act.nato.int/WISE

Note: BRITE Portal is part of this.

Page 19: NCOIC Geospatial Interoperability Task Team Presentation

19

How Would I Build a New App?

• Goal: Global Natural Disasters and Supply Chain Management

• Activities:– Use Previous Work Like Global Disasters Disaster

Knowledgebase– Build Google Search Index in Spreadsheets– Build Knowledgebases Based on the New Sitemap and

Schema Protocols– Federate the Distributed Knowledgebases for Analytics

Page 20: NCOIC Geospatial Interoperability Task Team Presentation

20

Global Natural Disasters Knowledgebase

http://semanticommunity.info/AOL_Government/2011_Natural_Disasters_Costliest_on_Record#Spotfire_Dashboard

Page 21: NCOIC Geospatial Interoperability Task Team Presentation

21

Google Search:Supply Chain Management

Pick The First One

Page 22: NCOIC Geospatial Interoperability Task Team Presentation

22

Google Search:Supply Chain Management Disaster Relief Databases

Pick The First One

Page 23: NCOIC Geospatial Interoperability Task Team Presentation

23

Google Search:Boeing Supply Chain Management

Pick The First One

Page 24: NCOIC Geospatial Interoperability Task Team Presentation

24

Google Search:Boeing Supply Chain Management

Note: Get 27 pages of results that could be indexed!

Page 25: NCOIC Geospatial Interoperability Task Team Presentation

25

Search:Spotfire Public Library

http://ondemand.spotfire.com/Public/Library.aspx?folder=/Public

Search for World Facts: Found Three Hits!Use the most recent one to add the NATOGeospatial Framework and Use Cases andSupply Chain Management Disaster Relief

Page 26: NCOIC Geospatial Interoperability Task Team Presentation

26

Knowledgebase Indicesin a Spreadsheet

http://semanticommunity.info/@api/deki/files/17448/=NATO.xlsx

Page 27: NCOIC Geospatial Interoperability Task Team Presentation

27

Spreadsheet in Desktop Dashboard App

The NGIA and NATO have questions about responding to new disasters, so they consult their knowledgbases of unstructured, structured, and complex data that have been provisioned, integrated, and visualized in this app.

Page 29: NCOIC Geospatial Interoperability Task Team Presentation

29

What Is a Hadoop?

http://www.saama.com/blog/bid/71286/What-is-a-Hadoop-Explaining-Big-Data-to-the-C-Suite

Page 30: NCOIC Geospatial Interoperability Task Team Presentation

30

What Is a Hadoop?• Hadoop is an emerging framework for Web 2.0 and enterprise businesses

who are dealing with data deluge challenges – store, process and analyze large amounts of data as part of their business requirements. Hadoop has rapidly emerged as a viable platform for Big Data analytics.

• Many experts believe Hadoop will subsume many of the data warehousing tasks presently done by traditional relational systems. This will be a huge shift in how IT apps are engineered.

• Take for instance, the competitive world of travel – airline, hotel, car rental, vacation rental etc.. Every site has to improve at analytics and machine learning as the contextual data is changing by the second – inventory, pricing, customer comments, peer recommendations, political/economic hotspots, natural disasters like earthquakes, etc. Without a sophisticated real-time analytics playbook, sites can become less relevant very quickly.

Page 31: NCOIC Geospatial Interoperability Task Team Presentation

31

What Is a Hadoop?

• Hadoop is designed to solve a different problem: the fast, reliable analysis of both structured, unstructured, and complex data.

• Hadoop consists of two elements: reliable data storage using the Hadoop Distributed File System (HDFS) and high-performance parallel/distributed data processing framework called MapReduce.

• Apache Hadoop = HDFS + MapReduce• MapReduce is the algorithm on which Google built its

empire!

Page 32: NCOIC Geospatial Interoperability Task Team Presentation

32

What Is a Hadoop?• Realworld examples: LinkedIn, CBS Interactive,

Explorys, and FourSquare, Walt Disney, Wal-mart, General Electric, Nokia, and Bank of America are applying Hadoop to a variety of tasks including marketing, advertising, and sentiment and risk analysis.

• IBM used the software as the engine for its Watson computer, which competed with the champions of TV game show Jeopardy.

• Many experts believe Hadoop will subsume many of the data warehousing tasks presently done by traditional relational systems. This will be a huge shift in how IT apps are engineered.

• 2012 may be the year Hadoop crosses into mainstream IT.

Page 33: NCOIC Geospatial Interoperability Task Team Presentation

33

Why Is What I Do Like Hadoop?

• Spotfire is designed to solve a different problem: the fast, reliable analysis and visualization of both structured, unstructured, and complex data.

• Spotfire consists of three elements: reliable data storage using advanced compression technologies, high-performance in-memory processing, and federated apps on mobile devices.

• Spotfire is what the defense and intelligence community uses along with pharma, oil and gas, etc. industries.

Page 34: NCOIC Geospatial Interoperability Task Team Presentation

34

Strata Conferences:Making Data Work App

• Strata 2012 Making Data Work: Introductory Video and All Videos.• Strata 2012 Making Data Work: Dave Campbell (Microsoft), "Do We Have The Tools

We Need To Navigate The New World Of Data?": Video I mentioned that talks about Haitian Disaster Response Translator and new way of doing business with big data.

• Strata 2012 Making Data Work: Mike Olson (Cloudera President), "Guns, Drugs and Oil: Attacking Big Problems with Big Data": Video I mentioned about what Cloudera is/wants to do for the Intelligence Community.

• Strata 2012 Making Data Work: Gary Lang (MarkLogic), “Why handing “big unstructured data” is important to the Intelligence Community”: Video I mentioned about making money at it.

• Strata 2011 Amazon’s CTO “Data Without Limits”: Video I mentioned about making a business of it.

• Note: Imagine the over 310 resources (titles, abstract, videos, slides, and papers) from all three conferences as linked data, organized by category and organization/sponsor, in a searchable database by facets, on the Web and your iPad! See the next slide.

Page 35: NCOIC Geospatial Interoperability Task Team Presentation

35

Strata Conferences:Making Data Work App

http://semanticommunity.info/AOL_Government/Data_Science_Visualizations_Past_Present_and_Future#Spotfire_Dashboard

Page 36: NCOIC Geospatial Interoperability Task Team Presentation

36

Strata Conferences:Making Data Work App

• Government, Real Estate, and Manufacturing had the highest value potential for big data.

• Still mostly methodology and technology and promise of applications except for $3M Health Prize (model of patient time in hospital).

• Next conference in connection with Hadoop World. Strata 2012 NY focuses on the big data issues that are shaping the business, finance, media, and government worlds.