Hybrid Data Pipeline for SQL and REST

New Data Pipeline Transforms How Clouds Access DataDion Picco, General Manager, Progress DataDirectSumit Sarkar, Chief Data Evangelist, Progress DataDirect

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.2

Audio Bridge Options & Question Submission


Twitter

Let’s Get Social

Sumit Sarkar, Chief Data Evangelist@SAsInSumit

Dion Picco, GM for Progress DataDirect @dpicco

Live Twitter Q&A #AskDataDirect

Connect With Progress DataDirect

LinkedIn

Facebook

http://app.businessmaking.progress.net/e/er?elq_n=DCI+Delivery_Oct12.html&s=1325&lid=9047&elqTrackId=8488326aff1f47e7b39383614f42a234&elq=59dd29f52115411987ac5f4645e2e62b&elqaid=10550&elqat=1

http://app.businessmaking.progress.net/e/er?elq_n=DCI+Delivery_Oct12.html&s=1325&lid=10092&elqTrackId=6a179704f10f49628f75a418302bb12d&elq=59dd29f52115411987ac5f4645e2e62b&elqaid=10550&elqat=1

http://app.businessmaking.progress.net/e/er?elq_n=DCI+Delivery_Oct12.html&s=1325&lid=9048&elqTrackId=aa58befdfa104d0ba946e85ff8f866b2&elq=59dd29f52115411987ac5f4645e2e62b&elqaid=10550&elqat=1

New Data Pipeline Transforms How Clouds Access DataDion Picco, General Manager, Progress DataDirectSumit Sarkar, Chief Data Evangelist, Progress DataDirect


During this webinar, you’ll learn:

Hybrid Clouds Pain and Pleasure

Introduce Hybrid Data Pipeline

How Hybrid Data Pipeline works to solve challenges with Hybrid Clouds


POLL #1 [Choose all that apply]

Which cloud infrastructure do you use today (select multiple)?• Amazon Web Services• Microsoft Azure• Salesforce• On-premises / Private Cloud• Other


Hybrid Clouds Pleasure and Pain


Rise of Hybrid Clouds

http://www.forbes.com/sites/louiscolumbus/2016/03/13/roundup-of-cloud-computing-forecasts-and-market-estimates-2016/#5b5f61a974b0

http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2016-state-cloud-survey#hybridcloudadoption

http://www.forbes.com/sites/louiscolumbus/2016/03/13/roundup-of-cloud-computing-forecasts-and-market-estimates-2016/#5b5f61a974b0


That Rise Brings New Cost Benefits

Scalability of on-premises infrastructure Reduce infrastructure and maintenance costs Capital-expenditure Free Models


That Rise Brings New Capabilities

Meet policy and regulatory security requirements• HIPAA• Data Privacy • PCI DSS• FISMA• SOX

Elastic capacity Rapid delivery of Omnichannel Apps Test and adopt the newest technologies (i.e. Machine Learning, Stream Analytics, Big

Data Storage, etc)


That Rise Also Signals the Demise of Secure and Easy Access to Data

Locked behind the firewall

Locked behind other clouds


For Data Behind a Firewall, There is no Common Access Approach for Clouds

1. Network Based VPN

2. SSH Tunneling

3. Reverse Proxy Servers


Introducing …


Introducing Progress® DataDirect® Hybrid Data Pipeline™(formerly “Project Mustang”)

Completely transform the way Cloud apps access dataProgress® DataDirect® Hybrid Data Pipeline is the industry’s first hybrid data pipeline that can run independently and integrate with any single or multi-vendor technology stack connected by open standards for SQL and REST.

Use Cases for Hybrid Data Pipeline


1. Firewall Friendly Connectivity

Lightweight and secure access to hybrid data sources over firewall friendly protocols –deployed as a standalone vendor agnostic component unlike heavyweight platforms or traditional connectivity not enabled for hybrid environments.

1. Firewall Friendly Connectivity success story in production

using DataDirect Cloud (Hosted Pipeline)


BOARD International

Business Intelligence (BI) and Corporate Performance Management (CPM) software vendor known for its BOARD toolkit.

“BOARD’s first customer to leverage the connector was a fashion retailer in Belgium, with more than 125 shops in Belgium and Luxembourg.

To date, six BOARD customers are leveraging the DataDirect connector, but Ferrari expects 75-80 percent of BOARD’s customers to purchase them over the next couple of years.”

Pietro FerrariChief Technology Officer

https://en.wikipedia.org/wiki/BOARD_International#Products


Intuit

Develops financial and tax preparation software and related services for small businesses, accountants and individuals.

“I wanted to know about implementation complexity. Lekhter told me that each rollout to a new user group took only a week of development. They push new business capabilities into production on a monthly basis. Lehkter credits the ease of development to the heavy lifting provided by Lightning Connect and Progress connectivity.”

Jerry LekhterDirector of Engineering


2. Single Interface to Cloud Data Sources

Reliable standard interface, for analytics and supporting architectures, to a variety of cloud APIs with varying rules for authentication, query, metadata, and qualities of service unlike the alternative of developing against each API version independently or requirement to sync data with an intermediary database.

2. Single Interface to Cloud Data Sources success story in production



Paxata

Paxata has deployed the DataDirect Hybrid JDBC client for their Data-Lake-In-The-Cloud customers running on Amazon Web Services. Customers seeking convenience of a hosted solution are clicks away from a customer data lake platform with self service access to cloud data sources across Marketo, Eloqua, Salesforce, Google Analytics, Oracle Service Cloud, Microsoft Dynamics, Hubspot and more through the single JDBC interface.

Paxata unveiled the first interactive, self-service Adaptive Data Preparation solution for business analysts, data scientists, developers, data curators and IT teams.www.paxata.com/company

Hosted Pipeline Client deployed in Paxata Cloud

http://www.paxata.com/company/


Chatham Financial

Result: Chatham Financial today has a data warehouse that meets stringent security standards, did not require internal resources to develop and deployed months faster when compared to using multiple vendors or building a custom solution.

With the financial services industry being highly competitive, Chatham Financial takes marketing seriously, continuously seeking ways to leverage its data to advance the business—but security always comes first. “Our clients expect top notch security from us, so we have to expect that from our vendors as well.”

Matt RobertsMarketing Analytics Manager


3. Expose your data source as standard SQL/REST

Expand connectivity options to include both standard SQL and REST from existing data sources across Big Data, EDW or Hosted Cloud Sources unlike alternatives that require developing and maintaining a scalable web services tier or intermediate SQL engine.

3. Expose data source as standard SQL/REST success story in production



ERP Vendor Moving to Cloud

Connectivity was restored for the cloud hosted application with DataDirect Hybrid Connectivity that enabled secure and firewall friendly access to the core ERP database. End customers were able to adopt the cloud ERP solution with standards based data access across:

1) SQL (ODBC) connectivity for BI tools

2) REST (OData) interfaces for clicks not code integration other clouds such as Salesforce

A leading global provider of enterprise software and information solutions migrated their business applications to the cloud, but end customers lost the ability to access data for reporting and analytics making the cloud unattractive


POLL #2 [Choose One Answer]

What is your biggest challenge with Hybrid Integration today?• Connect Salesforce with external data on-premises such as

SQL Server or Oracle• Connect my cloud or mobile app to on-premises data• Single API to many cloud and on-premises data sources• Managing network infrastructure (VPN, Reverse Proxies,

Firewall)• Other

Trusted Hybrid Data Pipeline

Hybrid Data Pipeline Architecture Diagram - PLACEHOLDER

On-Premises Data HostWindows Native EXE

HSQL DB

Java Process

Notification Server

Java Process

Data Access ServerWebUI

Tomcat Server, Java Process

TCP 9001JDBCSQL

TCP 8282Message Queue

On-Premises DAS

Tomcat Server,Java Process

DATATCP 10180 LAN

DB-specific

UDP 11235TCP 11280SSL 11443

HeartbeatsConnection Requests

DB

DB

On-Premises Connection

HandlerService API

DATASSL 40501

REST / SQLHTTP 8080HTTPS 8443 Firewall

ODBCJDBCOData

Clients

REST / SQLHTTP 8080HTTPS 8443

HTTP/SCloud Data

Sources

OPCOn-Premises

Connector

DASData Access Service


Progress Security Vulnerability Response Policy

Upon identification of any security vulnerability that would impact one or more Progress product(s), Progress will exercise commercially reasonable efforts to address the vulnerability in accordance with the following guidelines:


Engineered from the Award Winning DataDirect Cloud Service

• Hybrid Data Pipeline was developed from the DataDirect Cloud service running for 3 years

• 10,000+ users in production

How Hybrid Data Pipeline works


Configure Hybrid Data Pipeline Source


Configure On-Premises Connector (Only required when accessing data behind a firewall)

Firewall friendly and secure connection without network configuration or requirement to open any ports


Access Data Directly Over SQL or REST From the Cloud

All data is encrypted in transit and data is not persisted in the pipeline


Supported Data Sources for HDPApache Hadoop Hive 0.8.0 and higher Amazon EMR 2.1.4 and higher Amazon Redshift

Cloudera CDH update 4 and higher Hortonworks 1.3 and higher IBM BigInsights 3.0 and higherMapR 1.2 and higherPivotal HD 2.0.1 and higherDB2 V9.1, V9.5, V9.7, 9.8 for Linux, UNIX, Windows DB2 V8.x for LUWDB2 11 for z/OS* DB2 V10 for z/OS DB2 V9.1 for z/OS DB2 UDB V8.1 for z/OSDB2 I 7.1, 7.2* (DB2 UDB V7R1, V7R2 for iSeries)DB2 I 6.1 (DB2 UDB V6R1 for iSeries)DB2 for I 5/OS (DB2 UDB V5R4 for iSeries)Eloqua (Oracle Marketing Cloud)

Google Analytics

Greenplum 4, 4.1, 4.2, 4.3Greenplum 3.3Informix Dynamic Server 12.1*

Informix Dynamic Server 11.0, 11.5, 11.7Informix Dynamic Server 10.0Informix Dynamic Server 9.2, 9.3, 9.4Microsoft Dynamics CRM 2011 Rollup 16, 2013, 2015

Informix Dynamic Server 11.0, 11.5, 11.7Informix Dynamic Server 10.0Informix Dynamic Server 9.2, 9.3, 9.4Microsoft Dynamics CRM 2011 Rollup 16, 2013, 2015

Microsoft SQL Server 2014*Microsoft SQL Server 2012Microsoft SQL Server 2008 R1, R2Microsoft SQL Server 2005Microsoft SQL Server 2000 Desktop Engine (MSDE 2000) Microsoft SQL Server 2000Microsoft SQL Azure

MySQL Enterprise Edition 5.0, 5.1, 5.5, 5.6*Oracle 12c R1 (12.1)*Oracle 11g R1, R2 (11.1, 11.2)Oracle 10g R1, R2 (10.1, 10.2)Oracle 9i R1, R2 (9.0.1, 9.2)Oracle 8i R3 (8.1.7)Oracle Service Cloud

Oracle Sales Cloud

Pivotal HAWQ 1.1*, 1.2*PostgreSQL 9.0, 9.1, 9.2, 9.3, 9.4*PostgreSQL 8.2, 8.3, 8.4Progress OpenEdge 11.0, 11.1*, 11.2*, 11.3*, 11.4*Progress OpenEdge 10.1.x, 10.2.xProgress Rollbase 2.0 and higherSAP Adaptive Server Enterprise 16.0*SugarCRM 7.1.6 and higherSybase Adaptive Server Enterprise 15.0, 15.5, 15.7Sybase Adaptive Server Enterprise 12.0, 12.5, 12.5.xSybase Adaptive Server Enterprise 11.9Veeva CRM

Hybrid Data Pipeline Resources


Learn More About Progress DataDirect Hybrid Connectivity

https://prgress.co/hybrid

Firewall Friendly ConnectivityAround the web:Salesforce Developers guide to hybrid connectivity

Progress joins Oracle for OData reveal on stage at OpenWorld

Building Mobile applications with hybrid connectivity

Related Webinars:OData External Data Integration Strategies for SaaS [Guests: Microsoft and Oracle]Firewall Friendly Pipeline for Secure Data Access

Webinar: Integrate Oracle Data Sources with Salesforce Connect [Guest: Appirio]

Single Interface to Cloud Data SourcesAround the web:First ever OData NLP Hackathon with Silicon Valley startups

Future of SAP Business Objects Universe with hybrid connectivity

How Oracle Data Integration (ODI) leverages hybrid connectivity

Recommended integration solution for ODI and Oracle Marketing Cloud

Microstrategy community introduction to hybrid connectivity

SAP Lumira extension for hybrid connectivity

Related Webinars:How Big Data ISVs Get Marketing Data into Lakes [Guest: Ernst Young]Webinar: Binding Kendo UI to Any Data with OData [Guest: Chris Woodruff]

https://prgress.co/hybrid

https://developer.salesforce.com/page/Building_a_Data_Integration_Proof_of_Concept_Using_Lightning_Components

http://www.odata.org/blog/Next-major-SaaS-application-demonstrating-OData-support-OOW15/

http://www.odata.org/blog/Next-major-SaaS-application-demonstrating-OData-support-OOW15/

http://www.odata.org/blog/Enterprise-Mobile-Application-Development-OData/

http://forms.progress.com/wbr-odata-integration-for-saas

http://forms.progress.com/wbr-odata-integration-for-saas

https://www.progress.com/campaigns/datadirect/webinars/firewall-friendly-pipeline-for-secure-data-access

http://forms.progress.com/datadirect-wbr-Oracle_Data_Integration_Salesforce_Connect



https://www.progress.com/blogs/first-odata-hackathon-challenge-winner-at-apiworld

http://www.odata.org/blog/Recap-on-OData-session-for-SAP-Business-Objects-Universe/

http://www.ateam-oracle.com/a-universal-cloud-applications-adapter-for-odi/

https://blogs.oracle.com/dataintegration/entry/odi_12c_and_eloqua_using

https://community.microstrategy.com/t5/Tech-Corner/What-MSTR-developers-should-know-about-Cloud-Data-Sources/ba-p/253083

https://blogs.sap.com/2015/08/05/connect-sap-lumira-to-eloqua-marketo-google-analytics/

http://forms.progress.com/datadirect-wbr-SaaS-Marketing-Data-Lake-thankyou

http://forms.progress.com/datadirect-wbr-SaaS-Marketing-Data-Lake-thankyou

https://www.progress.com/campaigns/datadirect/webinars/on-demand-webinar-binding-kendoui-to-any-data-with-odata-archived-replay

https://www.progress.com/campaigns/datadirect/webinars/on-demand-webinar-binding-kendoui-to-any-data-with-odata-archived-replay

Questions ?

Hybrid Data Pipeline for SQL and REST

Technology

Transcript of Hybrid Data Pipeline for SQL and REST