Post on 30-Oct-2014
description
© 2007 IBM Corporation
Information Management Trendsand Some History
C. Mohan, PhD IBM Fellow & IBM India Chief Scientist Member, IBM Software Group, Asset Architecture & Information Management Architecture Boards
http://www.almaden.ibm.com/u/mohan/mohan@almaden.ibm.com
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan2
Key Customer Pain Points
Can’t Find Information – Discovery
Can’t combine Information – Integration
Can’t extract value from Information – Insight
Can’t consume Information – Dissemination
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan3
Today’s business challenges mandate a fresh approach to
managing information
Managing information in silos has
become obsolete
The Information Challenge Information is in Silos… Trusted Information is Not Available
Multiple Versions of the Truth
Inaccurate, Untimely
Inconsistent
Incomplete, Inaccessible
Out of Context…
Globalization, M&As
Risk & Compliance,
Eroding Customer Loyalty,
Supply Chain Complexity,
Industry Transformations,
Cost Cutting…
70% of people’s time can be spent searching for
relevant information
60%+ of CEOs: Need to do a better job leveraging
informationSources: IBM Attributes & Capabilities Study, 2005; Client Interviews 2004; IBM CFO Study, 2006
5X More Value creation by organizations effective at
using Information as an Asset
Information Must Become a
Strategic Asset
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan4
Information Management Trends
Information Intensive Applications Shift from transaction-centric to information-intensive applications
Information Diversity Delivering insight over increasingly diverse sources of information
New Business & Delivery Models Information as a Service, Outsourcing, New Licensing Models
Democratization of Information Changing User Expectations & the “Parent Test”
Massive Collaboration & Societal Intelligence Collaboration over shared information to creating business insight
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan5
Presentation Services
EDW
Legacy LegacyPortals, Browsers, and or Devices
StrategicAPPL
EventProcessing
TacticalAPPL
TxAPPL
AppServer
DiscoveryAPPL
MasterDataAPPLProcess
Services
Information Integration Services Analytic Services
Master Data Services
Transaction Application Services Analytic Application Services
Business Process Management
Federation
Discovery Services
ECW
Content ServicesCollaboration Services
Notes
Enterprise Service Bus
Metadata Services
Master data Hubs
Product Customer
Supplier Location
Transaction Services
OLTP2OLTP1
OLTP
BusinessRules
BusinessMonitoring
StreamingBatch
Metadata
Information as a Strategic Asset
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan6
Compliance & Risk Mgmt. Sales and
Marketing – Closed Loop
Campaign Mgmt.
CustomerService
Data Stewardship& Administration
Compliance
Marketing
AccountAdministration
Privacy Management
Web Self-Service
WirelessSelf-Service
Distributor
IVRSelf-Service
Branch / Sales Office
Call CenterBrowser-based
UnlimitedAttributes
MultipleCategorizations
Multi-enterprise
Standards-based
Security andAudit
NewBusiness
Processing
Privacyand Data
Mgmt.
MarketingInsight
CustomerFacing Channels Internal Users
Customer
Master Data
Master Data Integration
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan7
Data Services– Databases, Warehouses, Tools…
Content & Discovery Services– Content Mgmt. & Integration Services– Discovery Services…
Information Integration Services– Quality Services– Transformation Services– Federation Services– Metadata Services…
Information Accelerators– Master Data Management– Entity Analytics– Information Warehousing– Customizable Dashboards– Industry Data Models…
Information Delivered On DemandBased on Services Oriented Architecture
IBM Information Management SoftwareDelivering Value Beyond Traditional Repositories
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan8
XML Developer “I see a sophisticated XML repository that also supports SQL."
SQL Developer"I see a sophisticated
RDBMS that also supports XML."
Familiar Programming Models
OptimizedStorage Models
MatureServices
Familiar Tooling
OptimizedPerformance &
Scale
DB2 9 – A Pure XML, Relational Hybrid
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan9
Integration of XML & Relational Capabilities
DB2 SERVER
CLIENT SQL/XML
XQuery
DB2 Engine
XMLInterface
RelationalInterface Relational
XML
DB2 Storage:
DB2 Client /Customer Client Application
– Applications combine XML & relational data
– Native XML data type (server & client side)
– XML Capabilities in all DB2 components
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan10
XQuerySQL/XML
APIs/ClientXML Indexes
XML Schemasupport Native
Storage
XML Load
Import/Export
Native XML support in DB2 with more to comeSeamless integration with the relational world
New XML
Join Methods
Tools
And all the
relational stuff
DB2 V9 pureXML support
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan11
DB2 V9 pureXML support
XML as a native data type
Pure XML storage and indexing
XQuery and SQL/XML support
XML Schema Repository
Schema validation
Application Support (Java, C/C++, .NET, PHP, etc.)
Visual Tooling, Control Center Enhancements
Annotated schema shredding
DB2 Utilities: Import/Export, HADR, etc.
…and more
Secure and Resilient
Infrastructure for a New
Breed of Agile
Applications
DB2
9
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan12
Some of Our Info Mgmt Research Legacy
Invention of Relational Model/Technology & SQL Research prototypes
ƒ System R ƒ R* Distributed DBMSƒ Starburst Extensible Object-Relational DBMS ƒ Garlic Heterogeneous DBMS
Product Contributionsƒ Data sharing on DB2 390 Sysplex ƒ DB2 UDB Query Processor ƒ Intelligent Minerƒ Lotus Notes R5 Recoveryƒ Discovery Link & DB2 Information Integrator
6 IBM Fellows from team of < 50
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan13
Why We Have Experience with Customers
Over 2 decades of partnership with SWG Toronto & SVL– Incorporation of Starburst prototype into DB2– Component Owners of DB2 for LUW’s Query Compiler– Versions 2 – 5 (1992-1997)– Dealt with customer APARs, Visits, & Presentations
Responsible for many DB2 innovations– Query Graph Model (internal query representation, key to extensibility)
– Query ReWrite and Optimizer technology
– ARIES recovery and locking methods
– Triggers and Constraints
– Star Join and Hash Join
– Object-relational features
– Automatic Summary Tables (materialized views)
– Visual Explain
– Index Advisor
Respected for our vision– World-class publications in leading database conferences– Cognizant of industry trends
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan14
Leveraging Technology and People
IMS
Development
DB2
Development
IDS / U2
Development
Customer
Requirements
IBM
Products
IBM
Research
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan15
SVL DB2 UDB for z/OS & OS/390IMSBusiness IntelligenceContent ManagementDB2 EveryplaceRed BrickIcingTraditional AD Languages
Boeblingen DB2 Text ExtendersSAP/R3 EnablementIntelligent Miner for DataIntelligent Miner for Text
Somers
HawthorneAdvanced Technology
AlmadenAdvanced Technology
Menlo Park & OaklandIDSXPSJDBCVisionaryCloudscapeDatabladesObject Connect & TranslatorContent Management
India DB2 UDB ServiceBusiness IntelligenceIDS
AustinGBIS
Portland XPS & DB2
LenexaIDS
Boulder & DenverContent ManagementU2
Datablades
Boca Raton & MiamiEMMSLA Informix Support
Rochester DB2 UDB for AS/400
Toronto DB2 UDB for UNIX, Windows, & OS/2
IBM Information Management Teams
Beijing Information IntegrationDB2 for zOSContent Management DB2 and IMS tools
Las VegasEntity Analytics
Over 6000 employees worldwide
Yamato High Speed Inverted Index SearchBusiness IntelligenceContent Management
Hursley Enterprise Master DataSolutions
India Software Lab– 3000 employees– Broad range of skills – all SWG Brands– Linux Competency Center
DB2 Lab within ISL– 100+ developers – Lab based services teams – DB2, CM, BI
Other Resources– India Research Lab– Solution Porting Center– Education Center for IBM Software– IBM Academic Initiative
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan16
A Spectrum of Info Serving Requirements
Platform: Mobile Desktop Small Servers Large Servers
Data Size: Micro Compact Large Extremely Large
Workload: Batch Online Transactions Real-time Analysis Data Mining
Structure: Hierarchical Relational Multi-Value XML
OS: Symbian PalmOS Windows Linux Unix i5/OS z/OS
Scope: Embedded Intra-application Single application Multi-application
Support: None Web/E-mail Business hours 24x7
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan17
Products to Match the Spectrum of Data Serving Needs
DB2 Everyplace
OLTP
Relational
MobileEmbedded
LinuxPalmOSSymbian
Cloudscape
OLTP
Relational
Intra-App / Single-App
Java
IDS
OLTP
Relational
Intra-App / Single-App
AIX, etc.Linux
Windows
DB2
OLTP &Analysis
Relational & XML
Single / Multi-App
z/OSI5/OS
AIX, etc.Linux
Windows
IMS
OLTP
Hierarchical
Single / Multi-App
z/OS
U2
OLTP
Multi-Value
Intra-App / Single-App
AIX, etc.Linux
Windows
Superior capabilities across the spectrum of requirements
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan18
DB2 for z/OS
The power and function of an open, industry standard data server with zSeries’ industry leading availability, performance, and security
What it takes to be the industry’s most extreme data server
Continuous application availability measured in years Ability to process over 1B SQL transactions per hour Uninterrupted growth from 1 byte to over a peta-byte Serving 100s of applications for 100,000s of users US Government’s highest security classification (zSeries) Support for industry standards: XML, Web services, Java, C, COBOL Support for complex business applications: SAP, PeopleSoft, Siebel
Extreme qualities of service XML and Relational data server
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan19
Technology Evolution with Mainframe Specialty Engines
Internal Coupling Facility (ICF) 1997
Integrated Facility for Linux (IFL) 2001
IBM System z9 Integrated Information Processor (IBM zIIP) planned for 2006
System z9 Application Assist Processor (zAAP) 2004
Building on a strong track record of technology innovation with specialty engines, IBM intends to introduce the System z9 Integrated Information Processor
Support for new workloads and open standards
Designed to help improve resource optimization for eligible data workloads within the enterprise
Centralized data sharing across mainframes
Incorporation of JAVA into existing mainframe solutions
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan20
Data ChallengesVariety, Velocity, and Volume New composite applications
need data from multiple sources Consumers expect holistic,
personalized, and value-addedcontent
Relational, XML, packaged applications, content repositories, file systems all contain critical business information
Increasing emphasis on current data Real-time analytics
Business activity monitoring
Petabytes will be the measure ofavailable online data
All client interactions are important ( e.g., instant messages, audio records, web traffic,…)
Internet and intranet content
The world produces 250MB of information every year for every
man, woman and child on earth.
10-100GB100s GB - 1TB
1 - 20 GBs100s MB100s KB
1999
1s TB1s TB
100s TB100s TB
1s TB1s TB
10s GB10s GB
1s GB1s GB
2004
10X
100X
100X
1,000X
10,000X
Common Database SizesCommon Database Sizes
Transactions
Warehouses
Marts
Mobile
Pervasive37% CGR DiskGrowth ’96-’07
70,000 TB of TV and Radio contentin 2002 alone; 30% growth/year
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan21
Addressing the Changing Characteristics of Data
Actionability
Heterogeneity
Scale
Query
CCGAGTACCCAC
Satellite & Surveillance Images and Video
Gene Sequences
Transactions
Text and Web
Increasing need to manage and analyze new data types
Protein Folding
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan22
Research in Information and Interaction
Drive our leadership technologies for search, structured and unstructured information processing and analytics, natural language processing, and conversational and multimodal interaction, across multiple tiers of business activities in SWG products and solutions. Foster the exploitation of components with these leading research
technologies in IGS services offerings.
Conversational and Multimodal Interactions
UnstructuredInformation
Management
InformationManagement
Database
Synthesis
Information Integration
Metadata
Speech Recognition
CM
InformationRetrieval
NLP
Analytics
Video Analysis
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan23
Worlds of Structured & Unstructured Data Come Together
Analytical
Complexity
Collect
Store
Retrieve
Drill
Mine
ETL
Warehouse
SQL
OLAP
Cluster, Classify, ..
Crawl
ECM
Search
Navigate
Cluster, Classify, ..
Solutions
II
Structured Data Unstructured Data
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan24
Need for Business Intelligence
HIPAAHIPAA
Basel IIBasel IIPatriot ActPatriot Act
Sarbanes-OxleySarbanes-Oxley
Loyalty Profitability Buyer Behavior Targeted Offers
Loyalty Profitability Buyer Behavior Targeted Offers
Homeland SecurityHomeland Security
Internet Buzz Anti-Money
Laundering Border Control Crime Information
Internet Buzz Anti-Money
Laundering Border Control Crime Information
Globalization Business Controls Mergers and Acquisitions Supply Chain Efficiencies
Globalization Business Controls Mergers and Acquisitions Supply Chain Efficiencies
Capitalism and Its Troubles: A Survey of International Finance -May 24, 2002
Capitalism and Its Troubles: A Survey of International Finance -May 24, 2002
Accountability and ComplianceAccountability and Compliance Customer KnowledgeCustomer Knowledge
Preparing for terrorHow scared should you be?
Nov 28th 2002 From The Economist print edition
Preparing for terrorHow scared should you be?
Nov 28th 2002 From The Economist print edition
Business PerformanceBusiness Performance
Risk Management Fraud and Abuse Public Protection
Risk Management Fraud and Abuse Public Protection
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan25
SOA Reference Architecture
Business Innovation & Optimization Services
Dev
elo
pm
ent
Ser
vice
s
Integrated environment for design
and creation of solution
assets
Manage and secure services,
applications &
resources
Facilitates better decision-making with real-time business information
IT S
ervi
ceM
anag
emen
t
Infrastructure Services
Optimizes throughput, availability and performance
ESBFacilitates communication between services
Ap
ps
&
Info
As
setsPartner Services Business App Services Access Services
Connect with trading partners
Build on a robust, scaleable, and secure services environment
Facilitates interactions with existing information and application assets
Interaction Services Process Services Information Services
Enables collaboration between people,
processes & information
Orchestrate and automate business
processes
Manages diverse data and content in a
unified manner
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan26
Understand Information Assets and Link to Business Context Discover information
metadata Map information to
business processes Develop data &
content models
Compose Information Services Across Heterogeneous Sources Extract, federate & transform
heterogeneous information
Service Information Requests Deliver unified data
& content Deliver business
context Discover
relationships
Ensure Performance, Availability & Security Meet Service Levels
Define & Refine Information Management Rules & Policies Monitor information usage over time
Information as a Service The SOA Lifecycle Mapped to Information Needs
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan27
and more…
abc…DB2
IBM ContentManager Oraclexyz…
Heterogeneous Applications & Information
Insight
Information as a ServiceOptimize, Virtualize, Integrate, Accelerate
Data & Content
BusinessContext
InsightfulRelationships
Master Data, Entity Analytics, Decision Portals, Executive Dashboards,Industry Data Models
Extracted or Real-time
Standards-based
e.g., XQuery, JSR170, JDBC, Web Services...
Information as a ServiceMoving From a Project-Based to a Flexible Architecture (SOA)
Processes PeopleTools & Applications
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan28
Information Services for SOAUnprecedented Business Flexibility
Store Information DB2 Viper
Optimized XML storage
Virtualize Information Access WebSphere Information Server
Integrate Information WebSphere Information Server
Accelerate Master Information WebSphere Customer Center
WebSphere Product Center
IBM Entity Analytics
Industry Models
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan29
Industry Solutions Deliver Insight On Demand
Law Enforcement Crime Information
Warehouse Entity Resolution Anti Money
Laundering
Banking
Basel II and Banking Data Warehouse
Entity Resolution
Health Care
Aligned Clinical Environment
Retail
RFID
Retail Data Model
Telco
Telco Data Warehouse
Insurance
Customer Insight
IIW
Automotive
Quality Insight Early Warning
Life Sciences
Drug Discovery
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan30
OmniFind Key Technologies
ContentContentCrawling Scalable Web crawler Data Source crawlers Content Push
Parsing/Tokenizing
HTML/XML 200+ Doc Filters Advance Linguistic
SearchCollections
Categorization Taxonomy Rule-based
Annotation Text Analytics Plug-in
Indexing Global Analysis Static Ranking Store
Dynamic Ranking Fielded Search Dynamic Summary Parametric Search Spell Checking
Searching
Security
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan31
Content Management Portfolio Strategy
Capture, store, and manage all forms of content
Complete and scalable, content management functionality
Document management
Image management
Digital asset management
Report management
Web content management
Records management
Digital rights management
Email/Messaging archiving and management
Collaboration tools
…
Enterprise-scale business process management
Cross-portfolio, out-of-the-box integration
Rich, common client platform
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan32
IBM Content Management Platform Roadmap
4Q20041Q2005
20052006
…and Beyond
WebSphere Portal V5.1Embeds DB2 Content Manager Runtime Edition (JCR)
Records Manager V4.1.1A Dynamic RM Infrastructure
Workplace Web Content Management V2.0 Leveraging DB2 Content Manager and WebSphere Portal Framework
DB2 Content Manager V8.3Enhance Doc RoutingEnable BPMExtend Integration CapabilitiesSeamless RM
DB2 Document Manager V8.3Compliance/RMExtending Native Language Support
DB2 CommonStore V8.3Full-Text SearchSeamless RM
First Step ECM Unified ClientNew PortletsJ2EE Web ComponentsExtend to DPMExtend Document ManagementEmail/Messaging Archiving and Management EnhancementsPhysical Records ManagementVirtual Records ManagementWCM Leveraging Workplace and DB2 Content Manager Runtime (JCR)
Common Content RepositoryWorkplace Unified End-User Experience (Client)Event FrameworkIntegrated / Interoperable DPM/BPMExtended ECM Capabilities as Add-On FeaturesEnterprise JCRIBM CM SDKEnterprise Content Integration – JSR170DB2 Content Manager Runtime in ISV ApplicationsLDDM* Fully Supports JSR170
Autonomic CapabilitiesContent PreservationContent IntelligencePervasive Enablement…and More
* Lotus Domino Document Manager
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan33
Query Optimization
Industry-Leading Optimization Extensible – SQL to XQuery! Optimizes for Parallel
I/O accesses Within a node (SMP) Between nodes (MPP)
Powerful for complex OLAP & BI queries Industry-Strength Engineering Portable
Across HW & SW platforms Databases of 1 GB to > 300 TB
Continuing "technology pump" of improvements from Research
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan34
Unstructured Information Management Architecture
Common Research infrastructure for advancing Text Analysis and NLP capability Promotes re-use of best-of-breed components Promotes combination hypothesis through ease of integration
Unstructured Information
Application Libraries
Specialized Application Libraries
Provide basic functions common to a broad class of application libraries & applications (e.g. Glossary Extraction Taxonomy Generation, Classification, Translation, etc.)
Question Answering
e-Commerce
Semantic Search EngineToken and Concept Indexing
Query Key words, concepts, spans, ranges -> Ranked Hit List
National & Intelligence Business
Bioinformatics
Technical Support
Document & Meta Data StoreDocuments with meta data based on key-value pairs
Enables view & collection management
(Text) Analysis Engine (TAEs)Combination of analysis engines employing a variety of analytical techniques and strategies
Structured Knowledge AccessKnowledge Source Adapters - (KSAs) deliver content from many structured knowledge sources according to central ontologies
Collection
Processing Manager
KSA Directory Service
Dynamic query & delivery of KSAs
TAE Directory Service
Dynamic query & delivery of TAEs
UIMA Standard Application Libraries
Relevant Application Knowledge
Structured Data
UIM
So
luti
on
s
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan35
Analytics bridge the Unstructured & Structured worlds
UnstructuredInformation
UnstructuredInformation UIMAUIMA
High-ValueMost Current ContentFastest GrowingBUT ...
Buried in Huge Volumes – Lots of NoiseImplicit SemanticsInefficient Search
Explicit StructureExplicit SemanticsEfficient SearchFocused Content
Text, Chat, Email, Audio,
Video
Text, Chat, Email, Audio,
Video
IndicesIndices
DBsDBs
KBsKBs
Identify Semantic Entities, Induce StructureChats, Phone Calls, Transfers People, Places, Org, Events Times, Topics, Opinions, RelationshipsThreats, Plots, etc.
Identify Semantic Entities, Induce StructureChats, Phone Calls, Transfers People, Places, Org, Events Times, Topics, Opinions, RelationshipsThreats, Plots, etc.
UIMA - The Big Picture
StructuredInformation
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan36
Evolution of Metadata
Hierarchical Data Model Rigid MetadataSingle Application
Domain Specific OntologiesFlexible MetadataCross Industry Integration
Increased Business Value of Metadata
Syntactic annotation of
data: what this data
represents
Semantic annotations of data: what this
data means
Relational Data ModelRigid MetadataIntegration Within Enterprise
Extensible Data Model (XML)Flexible MetadataIntegration Within Industry
1970 1990 2000 20101980
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan37
Data-driven analysis, reporting, monitoring, data rule & integration
specification
Data Analysts
Business context mapped to information
technology assets
Subject Matter Experts, Data
Stewards
Simplify integration
Metadata and data-driven data modeling
and management
Architects
Increase trust and confidence in information
Increase compliance to standards
Facilitate change management & reuse
Database application and transformation
development
ImplementersData
Administrators
Development Data Modeling Data Stewardship
Metadata Server
Integrated Metadata Enables Shared Understanding
Business Glossary
Data Architect Source System AnalysisInformation Analyzer
DataStage
QualityStage
Information Server
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan38
How Does Metadata Make Information Services Different?
Getcustomer
Getcustomer
OtherData Sources
ContentRepositories
?
WSDL WSDL
Information Services provide a basis for trust in information – providing visibility into lineage, relationships to other systems, and business definition
Traditional Service Information Service
• Where does the information come from?• What happens to it along the way?• How does this fit into how the business defines things?• How do I know I’m using the right service?
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan39
Metadata-driven Design for Integration
40% of IT budgets may be spent on integration
30% of people’s time is searching for relevant information
30% of development time is copy management
Remember ItRemember relationships and dependencies
Find ItFind and visualize related information
Connect ItGenerate the integration glue
WebService
Build These
Using These
New Business Process
New Integrated View
Legacy and packaged apps
Relational databases
XML documents
New DataFlow
WBI II ETL
DBA Mindshare, Kolkata 26 Apr 2007, C. MohanDBA Mindshare, Kolkata 26 Apr 2007, C. Mohan40
Metadata Will Be Used to Facilitate Information and Application IntegrationToday – manual
integration, custom hard-wired integration
Tomorrow – semi-automated integration by using tools and connectors
Future – automated integration through metadata standards and tools
Dictionaries
Taxonomies
Ontologies