Denodo Data Virtualization Platform: Overview (session 1 from Architect to Architect webinar series)
-
Upload
denodo -
Category
Data & Analytics
-
view
448 -
download
5
description
Transcript of Denodo Data Virtualization Platform: Overview (session 1 from Architect to Architect webinar series)
Five In-depth Technology and Architecture Sessions on Data Virtualization
Session 1: Denodo Platform – Architecture
Today’s Speaker
■ Paul Moxon
Senior Director, Product Management
Architect-to-Architect Series
■ Series of five webinars over next 2 months
■ Deeper look into Denodo Platform
■ Architectural Overview (today’s sessions)
■ Performance
■ Scalability
■ Data Discovery and Governance
■ Security
Denodo Express
■ Denodo Express
■ Free to Download
■ Fully functioning Data Virtualization Platform
■ Single user, supports common data sources
■ Many of the same capabilities of Denodo Platform
■ Performance, Data Discovery, Governance, internal Security, Publishing, …
• Intuitive graphical design tool
• Drag-and-drop, point-and-click modeling and integration
• Connectivity to all types of data sources
• Native support for hierarchical data structures via ERM
• Extensive on-the-fly data transformation & data cleansing and matching functions
• Multiple ‘publishing’ protocols
• Metadata exploration
• Data lineage, dependency analysis, change impact analysis, etc.
• Global Search
• Search data and metadata across all data sources
• Team/distributed development models
• Version Control System integration …
Denodo Platform – Design time
• Proven scalability & performance
• Scales from departmental/project to enterprise-wide with geographically distributed nodes
• Handles largest transactional (users, transactions, or sources) and analytical loads (data volumes)
• Advanced query optimizer
• Cost-based & source constraint-based query optimization incl. query delegation, query rewrite, join optimization, data movement, …
• Intelligent caching
• Full, partial, time-based caching
• Fine-grained security model
• Integrate with external systems
• Enterprise–grade monitoring and auditing
• Integration with System Management products e.g. HP BTO, Unicenter, Tivoli, etc.
• Full development & deployment lifecycle support
Denodo Platform - Runtime
Denodo Platform – Detailed Architecture
Performance – Architecture Modules
Performance Optimization
■ Advanced Query Optimization
■ Cost and Source Constraint Based Query Plans
■ Query Delegation
■ Automatic Query Rewriting
■ Join Optimizations
■ Data Movement
■ Asynchronous Multi-threaded Processing
■ Server Throttling Mechanisms
■ Scalability
■ Caching
■ Multiple configuration modes – full or partial
Performance – Optimizer, etc.
■ Optimizer
■ The Optimizer applies state-of-the-art optimization techniques to relational and non-relational sources.
■ Query Plan Generator
■ The Plan Generator is in charge of generating possible execution plans for the query and selecting the optimum one.
■ Execution Engine
■ Responsible for executing the selected query plan, executing the necessary sub-queries on the sources (or collecting data from cache as appropriate) and integrating the results to generate the global response.
Scalability – Architecture Modules
Scalability – HA and Clustering
■ High Availability Architecture
■ Active-Passive
■ Active-Active
■ Vertical Scalability
■ VM Support
■ Metadata Sharing
■ Horizontal Scalability
■ Clustering
■ Shared Cache or Individual Cache
Scalability – HA and Clustering
■ Denodo Cache
■ The Denodo Platform incorporates an advanced cache system to improve performance
■ For data materialization it makes use of an external JDBC database (physical or in-memory)
■ Denodo High-Availability
■ Set of features provided by the Denodo Platform to support High Availability and Load Balancing
■ Denodo Cluster Management
■ Tool for monitoring and managing a cluster of Denodo servers
Data Discovery and Governance – Architecture Modules
Data Discovery and Governance
■ Metadata Repository
■ Apache Derby database
■ Contains view and data source configuration data
■ Relationships between views – dependencies, etc.
■ Searchable – Catalog searches, etc.
■ Data Lineage
■ Trace how data changes between source and consumer
■ Change Impact Analysis
■ What is the impact of a change in a data source schema?
Data Discovery and Governance
■ RESTful Web Services
■ Supports Global Search functionality
■ Index and search data sources
■ ‘Google’ like search
Security – Architecture Modules
Security
■ Authentication & Authorization
■ Built-in User/Role Management Module
■ Integration with external entitlement servers (LDAP/AD)
■ Multi-level access controls
■ Database, View, Row, Column, and Cell
■ Policy-based Security and Workload Management
■ Enforcement of custom policies for query execution according to security / workload considerations
Extensibility – Architecture Modules
Extensibility
■ Stored Procedures
■ Extension point for custom behavior (based on a Java SDK)
■ Eclipse Plugin for ease-of-development
■ Custom Functions
■ Extension point for custom transformation and data quality functions
■ Custom Wrappers
■ Extension point for custom data sources
■ Denodo API
■ Integration point for external tools and utilities
Q&A
Data Virtualization – Next Steps
Move forward at your own pace
Download Denodo Express –
The fastest way to Data Virtualization
Denodo Community: Documents, Videos, Tutorials, more.
Attend Architect-to-Architect Series
Performance
Scalability
Move forward with one of our Data Virtualization experts
Phone: (+1) 877-556-2531 (NA)
Phone: (+44) (0)20 7869 8053 (EMEA)
Email: [email protected] | www.denodo.com
Data Discovery and Governance
Security
Five In-depth Technology and Architecture Sessions
on Data Virtualization
Thank You!
Next Session
Session 2 Denodo Platform: Performance