Fusing Structured and Unstructured Data for Geospatial Insights in Lumify
-
Upload
charlie-greenbacker -
Category
Data & Analytics
-
view
108 -
download
1
description
Transcript of Fusing Structured and Unstructured Data for Geospatial Insights in Lumify
Fusing Structured and Unstructured Data for Geospatial Insights in
Charlie Greenbacker Susan Feng Altamira Technologies Corporation
is an open source big data analysis and visualization platform built by Altamira engineers
Key Lumify Concepts
structure for organizing information (i.e., your data model) Ontology
any “thing” you want to represent (e.g., person, place, event) Entities
a link between two entities (e.g., leader-of, works-for, sibling-of) Relationships
data about an entity (e.g., first name, last name, date of birth) Properties
collection of entities and the relationships between them Graph
What you can do with
trafficking
RESULTS
Document 94
FILTER BY ENTITY PROPERTIES
GEO LOCATION REMOVE
Latitude 23.22
Longitude -106.42
Radius 1000
DATE REMOVE
is between 2014-01-01
2014-03-01
ADD FILTER
Video 27 Image 39 Event 21
Raid 21
Drug Lord 25
Person 60
Politician 35
Lumify provides full-text search over everything in your graph. Use custom filters built from properties defined in your ontology to refine your search.
Search
Joaquin Guzman Loera
Display related entities, find paths to another entity, and establish new relationships to other entities all from a right-click menu or drag and drop action.
Link Analysis
Connect…
Find Path…
Search Related
Remove Remove from workspace
^
^
Add Related… Items
Raw
^R
Documents
Images
Videos
People
Contact Information
Organizations
Events
Locations
Lumify provides many different ways to resolve new entities, establish relationships, and assign properties from the details view, map, or graph.
Knowledge Building
Zarka de Mexico Joaquin Guzman Loera
617-589-9821
Joaquin Guzman…
works at
owns
founded
advises
The graph leverages drag-and-drop and context menus to put common actions at your fingertips. Use auto layout options to tame large graphs.
Graph Visualization
2014-02-10 +52 1 825 5536872 +52 1 877 1211498
303-301-5881
303-904-7511
Mazatlan
Mexico City
2014-02-22 2014-02-22
Joaquin Guzman… Zarka de Mexico
Emma Coronel Patraca
Ismael Garcia
Javier Felix
Lumify ingests unstructured text documents, images, video, and audio files, then uses a variety of tools to extract & enrich the data for discoverability, analysis, and visualization.
Multimedia Analysis
Drug Lord “El Chapo” Captured in Mexico
PUBLISHED DATE
SOURCE
Audit
2014/02/22 Wikipedia
Add Property
Although Guzman had long hidden successfully in remote areas of the Sierra Madre mountains, the arrested members of his security team told the military he had begun venturing out to Culiacan and the beach town of Mazatlan. A week prior to his capture, Guzman and Zambada were reported to have attended a family reunion in Sinaloa. The Mexican military followed the bodyguards tips to Guzman’s ex-wife’s house, but they had trouble ramming the steel-reinforced front door, which allowed Guzman to escape through a system of secret tunnels that connected six houses, eventually moving south to Mazatlan. He planned to stay a few days in Mazatlan to see his twin baby daughters before retreating to the mountains. On 22 February 2014, at around 6:40 a.m., Mexican authorities arrested Guzman at a hotel in a beach front area in Mazatlan, Sinaloa, following an operation by the Mexican Navy, with joint intelligence from the DEA and
Geo-tagged data can be aggregated and viewed using any mapping system with support for OpenLayers, including ESRI and Google Maps.
Geospatial Analysis
Geospatial data in
Sources of Geospatial Data in Lumify
geotags & coords in database records, metadata, etc. Structured Data
location fields & addresses in spreadsheets, etc. Semi-structured Data
place names mentioned in text documents Unstructured Data
CLAVIN: an open source geoparser
geotagging & parsing of unstructured text Turns Text into Maps
resolves place names to gazetteer records Geospatial Entity Resolution
solves the “Springfield problem” Disambiguation
now handles multipart location fields (e.g., [Reston|VA|US]) Versatile
created by Berico Technologies www.clavin.io
How does CLAVIN work?
(i.e., machine learning + natural language processing)
demo
Who can
help?
Lumify helps analysts fuse structured and unstructured data from myriad sources into actionable intelligence.
Intelligence Analyst
Law enforcement personnel can use Lumify to explore criminal networks, uncover hidden connections, and develop leads.
Police Investigator
Lumify analyzes financial data and transaction records to help detect fraud and identify possible insider threats.
Financial Analyst
photo credit: “Numbers And Finance” by Ken Teegardin (h<ps://flic.kr/p/9rn9Yh) CC-‐BY-‐SA 2.0
Scientists, law firms, news organizations, and others can track their research in Lumify to unearth latent knowledge and discover critical new insights.
Research Staff
photo credit: “A researcher at The NaJonal Archives in Kew” by the UK NaJonal Archives (h<p://bit.ly/1n9dhR8) CC-‐BY 3.0
Built on Scalable Open Source Tech
Hadoop CDH 4
Accumulo
ElasJcSearch
tesseract CLAVIN CMU Sphinx OpenNLP OpenCV ffmpeg
Apache Storm
Secure Graph
custom code
Questions?
www.lumify.io
try.lumify.io
@lumifyio