Semantically integrated Enterprise Data Lakes and Co-Evolution of Public / Private Data

WWW.LEDS-PROJEKT.DE

LINKED ENTERPRISE DATA SERVICES

DR. MICHAEL MARTINUNIVERSITÄT LEIPZIG / AKSW

8. November

• Forschungseinrichtungen:• Universität Leipzig / AKSW

Forschungsgruppe• Technische Universität Chemnitz

• Firmen:• brox IT-Solutions GmbH• Ontos GmbH• Netresearch GmbH & Co. KG• Lecos GmbH• eccenca GmbH

PROJEKTÜBERSICHT

November 8, 2016

LEDSFORSCHUNGSGEBIETE

November 8, 2016

• Open eGovernment Data

• Semantic eCommerce

• Natural Language Processing

• Linking and Knowledge Extraction

• Versioning and Co-Evolution

• Big Data / Linked Data Integration

November 8, 2016

FORSCHUNGSGEBIETE

November 8, 2016

BusinessUnit2 BusinessUnit3 BusinessUnit4 BusinessUnit5BusinessUnit1

CorporateMemory

Inbound

DataSources

OutboundandConsumption

InboundRawDataStore

BigData DWH-Infrastructure

KnowledgeGraphforMetaData,KPIDefinitionand DataModels

FrontendtoAccessRelationshipandKPIDefinition/Documentation FrontendtoAccess(adhoc)Reports

OutboundDataDeliverytoTarget

Systems

WWW.LEDS-PROJEKT.DE

ECCENCA CORPORATE MEMORY

SEMANTICALLY INTEGRATED ENTERPRISE DATA LAKES

RENE PIETZSCH

8. November

LEDSMOTIVATION

Enterprise Data Management Objective:

“Ensure all data is aligned to a common meaning in order to achieve automation in performing complex analytics and generating trusted reports.”

Source:2015DataManagementIndustryBenchmark- EDMCouncil

November 8, 2016

In 2015 only 7% of respondents claim to already be using shared and unambiguous definitions of data across the firm and have it accessible as operational metadata.

©eccencaGmbH2016

Accounting RegulatoryReporting

RiskMgmt. Treasury...

PerspectivesonDataturnintosilosofdatabeingduplicated,annotated,simplychangedovertime,makingreconciliationandinterpretationachallenge

MOTIVATION

©eccencaGmbH2016

LEDSARCHITECTURE

November 8, 2016

ManagementAccounting

RiskManagementRegulatoryReporting

Treasury MarketingAccounting

CorporateMemory

Inbound

DataSources

OutboundandConsumption

InboundRawDataStore

KnowledgeGraphforMetaData,KPIDefinitionandDataModels

FrontendtoAccessRelationshipandKPIDefinition/Documentation FrontendtoAccess(adhoc)Reports OutboundDataDeliveryto

TargetSystems

©eccencaGmbH2016

LEDSARCHITECTURE

InboundRawDataStore

TargetSystems

DataIngestion• Filesinthedatalake(CSV,XML,Excel)• (relational)Databases

©eccencaGmbH2016

LEDSARCHITECTURE

InboundRawDataStore

TargetSystems

BigData

DWH-Infrastructure

DataLake• Emergingapproachtohandlelargeamounts

ofdata• Cost-effectivestorage• DataisheldintheirnativeformatsGoodDoesnotforceanup-frontintegrationoftheingesteddatasetsBadRetaininganoverviewofdisparatedatasilosinthelakewithouthavingacoherentsharedviewisachallengingissue

DataWarehouses• Existinginfrastucture• Typicallyrelationaldatabases

LEDSARCHITECTURE

InboundRawDataStore

TargetSystems

Metadata Layer• DatasetMetadata• Ontologies• IntegrationRules

Graphical UserInterface

CustomerApplications

LEDSDATASET MANAGEMENT

DatasetManagement• CatalogDatasets• CatalogOntologies• ManageMetadata

DatasetDiscovery• DataProfiling• DatasetExploration

DatasetIntegration• DatasetLifting• DatasetLinking• DataQualityValidation

DataAccess• DomainSpecificConsolidatedViews

• ExecutiononHadoop

November 8, 2016

LEDSDATASET DISCOVERY

November 8, 2016

LEDSINTEGRATION PROCESS 1/2

November 8, 2016

LEDSINTEGRATION PROCESS 2/2

November 8, 2016

LEDSDATA ACCESS

ContactRenePietzschTel:+491726940915email:rene.pietzsch@eccenca.com

eccencaCommand your Data!

Semantically integrated Enterprise Data Lakes and Co-Evolution of Public / Private Data

Software

Transcript of Semantically integrated Enterprise Data Lakes and Co-Evolution of Public / Private Data

A semantically enabled architecture for crowdsourced Linked ...A semantically enabled architecture for crowdsourced Linked Data management Elena Simperl Institute AIFB Karlsruhe Institute

Semantically Mapping Science: Results · create a Science Barometer. A robust tool for studying scientific activity as it happens through the use of web data sources. Semantically

TrueWeb A Proposal for Scalable Semantically-Guided Data ... · TrueWeb Overview •Create a semantically-guided system for knowledge graphs •Create reliable truth finding techniques

LNCS 8219 - A Linked-Data-Driven and Semantically-Enabled ...nutt/Teaching/SemTechs1415/Sem... · A Linked-Data-Driven and Semantically-Enabled Journal Portal for Scientometrics YingjieHu

ifcWOD (Web Of Data) - Semantically Adapting IFC Model Relations into OWL Properties

Logical Data Warehouse and Data Lakes

Semantically supporting data discovery, markup and aggregation in EMODnet

semantically formalized logs

Semantically-enabled (large-scale) Scientific Data Integration (SESDI)

Semantically Incorrect

A Semantically Enabled Service Architecture for Mashups over Streaming and Stored Data

Web Usage Mining Structuring semantically enriched clickstream data

Introducing Data Lakes

A semantically adaptable integrated visualization and ... · A semantically adaptable integrated visualization and natural exploration of multi-scale biomedical data Ricardo Manuel

Certificate Big Data for Data Lakes

Learning Semantically Robust Rules from Data · Learning Semantically Robust Rules from Data Yiheng Li Latanya Sweeney February 2004 CMU-CALD-04-100 ... traditional association rule

Reconstructing the Chaine operatoire through Semantically Linked Open Data

A Linked-Data-driven and Semantically-enabled …geog.ucsb.edu/~jano/sejp.pdfA Linked-Data-driven and Semantically-enabled Journal Portal for Scientometrics ... Linked Data is a ...

Data Lakes: 8 Enterprise Data Management Requirements

Learning Semantically Robust Rules from Data