VerticaKnowledgeBaseArticle VerticaIntegrationwith ... · Vertica Integration with Microsoft Power...

Vertica Knowledge Base Article

Vertica Integration withMicrosoft Power BI:Connection Guide

Document Release Date: 3/1/2019

Legal Notices

WarrantyThe only warranties for Micro Focus International plc products and services are set forth in theexpress warranty statements accompanying such products and services. Nothing herein shouldbe construed as constituting an additional warranty. Micro Focus | shall not be liable fortechnical or editorial errors or omissions contained herein.

The information contained herein is subject to change without notice.

Restricted Rights LegendConfidential computer software. Valid license from Micro Focus | required for possession, useor copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, ComputerSoftware Documentation, and Technical Data for Commercial Items are licensed to the U.S.Government under vendor's standard commercial license.

Copyright Notice© Copyright 2015-2019 Micro Focus International plc

Trademark NoticesAdobe™ is a trademark of Adobe Systems Incorporated.

Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation.

UNIX® is a registered trademark of The Open Group.

This product includes an interface of the 'zlib' general purpose compression library, which isCopyright © 1995-2002 Jean-loup Gailly and Mark Adler.

Micro Focus | Vertica of 26

Contents

Vertica Integration with Microsoft Power BI: Connection Guide 4

About Vertica Connection Guides 4

Vertica and Power BI: Latest Versions Tested 4

Power BI Overview 4

Before You Start 5

Download and Install Power BI Desktop 5

Power BI Connection Modes - DirectQuery vs Import 5

Connect to Vertica from Power BI Desktop 6Connecting to Vertica via the Vertica Named Connector 6Connecting to Vertica via a Generic ODBC Connection 10

Connect to Vertica from Power BI Service 16The Power BI On-Premises Data Gateway 16

Known Issues and Work-Arounds 22Publishing to Power BI Service fails using DirectQuery 22DirectQuery doesn’t support DAX functions/transformations 23String Data Too Big Error 24

Data Type Limitations 25

For More Information 26

Vertica Integration with Microsoft Power BI: Connection Guide


Vertica Integration with Microsoft Power BI:Connection GuideAbout Vertica Connection GuidesVertica connection guides provide basic instructions for connecting a third-party partnerproduct to Vertica. Connection guides are based on our testing with specific versions of Verticaand the partner product.

Vertica and Power BI: Latest Versions TestedThis document is based on our testing using the following versions:

Software Version

PartnerProduct

Power BI Desktop version 2.63 64-bit (October 2018 release)

Power BI On-premises Data Gateway (October release build number3000.0.144.3)

DesktopPlatform

Windows Server 2012

Vertica Client Vertica 7.2 ODBC driver (Power BI Vertica connector)

Vertica 9.1 ODBC driver (generic ODBC connection)

Vertica Server Vertica 9.1

Power BI OverviewPower BI is a business intelligence solution by Microsoft Corporation that provides tools toanalyze data and build interactive dashboards that can be made available in the cloud and onmobile devices. Power BI consists of the following components:

l A Windows desktop application for exploring your data and building reports. You can publishPower BI Desktop reports to the web and share them with others via Power BI Service.

l Power BI Service is a Software as a Service product for viewing and sharing data reports thatyou build using Power BI Desktop. To use the Power BI Service, you need a web browser andan email address.

Power BI Service is also known as Power BI Server and Power BI Site.

l Power BI Mobile provides apps for Android, iOS, and Windows phones. These apps allow youto view reports published on Power BI Service on your mobile device.



l Power BI Report Server is the on-premises version of Power BI Service. It is used in caseswhen the customer wants to host the Server on their own infrastructure. It requires a SQLServer Report Server instance and it has a different release cycle than Power BI Service.Also, it uses a separate install of Power BI Desktop that is optimized for Report Server.

For a comparison between Power BI Service and Power BI Report Server, read this blog:https://docs.microsoft.com/en-us/power-bi/report-server/compare-report-server-service

To download Power BI, go to https://powerbi.microsoft.com/en-us/downloads/.

Before You StartBefore you connect to Vertica using Power BI Desktop, you must:

l Install and start Vertica. If you have not installed Vertica, go to the Vertica download pageand download the Vertica Community Edition.

l Deploy the VMart example database. The examples in this document use VMart.

For information about VMart, see Introducing the VMart Example Database in the Verticadocumentation.

Download and Install Power BI DesktopTo download Power BI Desktop, go to Microsoft Power BI Downloads.

Under Microsoft Power BI Desktop, if you click Download, you download the 64-bit version.To download and install the 32-bit version, click Advanced download options and follow theinstructions.

After the download completes, execute the downloaded file and follow the instructions.

Power BI Connection Modes - DirectQuery vs ImportPower BI connects to Vertica using Vertica’s ODBC driver and accesses data in one of twoways:

l DirectQuery connection mode: This connection type enables push down of the queriesgenerated by Power BI reports to your Vertica database, and only transfers/imports theresult of your queries into Power BI. After connecting, the data remains on the Verticaserver.

For details about DirectQuery, see the following topics in the Power BI documentation:

l Use DirectQuery in Power BI Desktop

l Data sources supported by DirectQuery in Power BI

l Import connection mode: When you use this type of connection, Power BI Desktoptransfers the data from Vertica into the Power BI Desktop cache. When you create orinteract with a visualization, Power BI Desktop uses the imported data to render thevisualizations. Imported data needs to be refreshed on a regular basis to reflect the latestchanges in the database.



https://docs.microsoft.com/en-us/power-bi/report-server/compare-report-server-service

https://powerbi.microsoft.com/en-us/downloads/

https://vertica.com/download/vertica/

https://vertica.com/docs/latest/HTML/index.htm#Authoring/GettingStartedGuide/IntroducingVMart/IntroducingVMart.htm


https://docs.microsoft.com/en-us/power-bi/desktop-use-directquery

https://docs.microsoft.com/en-us/power-bi/desktop-directquery-data-sources

Connect to Vertica from Power BI DesktopCurrently Power BI provides two options for connecting to your Vertica database:

l Option 1: New Vertica named connector

Previously in beta, the Vertica named connector is now generally available.

l Allows you to build DirectQuery-based and import-based reports against your Verticadatabase. For information about DirectQuery and import, see Power BI Connection Modes- DirectQuery vs Import in this guide.

l Vertica’s ODBC driver ships with the new Vertica named connector. You don’t need todownload and install Vertica’s ODBC driver on the client machine.

l DirectQuery does not support DAX functions/transformations. DAX functions may berequired to model the data and build the reports.

l DirectQuery is supported from Power BI Service through the on-premises data gateway.

Important

To refresh datasets based on Vertica data in the Power BI Service, you need toupgrade to the October 2018 release of the on-premises data gateway.

l Option 2: Generic ODBC connection

l Allows you to build import-based reports against your Vertica database. For informationabout import connection mode, see Power BI Connection Modes - DirectQuery vs Importin this guide.

l You must download and install Vertica’s ODBC driver and create an ODBC DSN thatPower BI uses to connect to Vertica.

l You must schedule periodic refreshes in Power BI Service over a gateway in order to seethe latest changes in the database.

Connecting to Vertica via the Vertica Named ConnectorPower BI Desktop now bundles the Vertica ODBC driver with the application. You no longerneed to download and install the Vertica ODBC driver on your Power BI machine unless you areusing the generic ODBC connector to connect to Vertica. The new native connector will not useor conflict with an existing Vertica ODBC driver installed previously on the system.

The ODBC driver that is shipped with Power BI is version 7.2.x.

Power BI allows Import as well as DirectQuery connection to Vertica. DirectQuery enables pushdown to Vertica. This lets you work with large volumes of data and leverage the speed ofVertica. For more information about DirectQuery connectivity, see Power BI Connection Modes- DirectQuery vs Import.



Note

By design, the Vertica named connector does not allow writing SQL statements. Thisoption is only available with Import mode using a generic ODBC connection. WritingSQL statements is highly recommended when using a generic ODBC connection tonarrow down the data transfer into Power BI and to improve performance whenworking with large datasets.

Connect to Vertica from Power BI Desktop using the Vertica connector as follows:

1. Open Power BI Desktop.

2. Click the Get Data icon in the home menu.

3. Type Vertica in the search input box or select it from Other in the list of connectors:



4. Click Connect.

5. In the connection window, enter the connection information for your Vertica database:

6. Select the Data Connectivity mode. Import is selected by default.

7. Click OK.

8. If prompted, enter your database user name and password:

9. Click Connect.

10. In the Navigator window, expand the database name to list the schemas.

11. Expand the schemas you are interested in and select the tables to use for your analysis.

In this example, you are connected to the VMart database. From the Public schema, selectthe inventory_fact table and its related dimension tables:



12. Click Load.

For DirectQuery mode, the status of the load operation displays as follows:

Evaluating...Waiting for other queries...Creating connection in model...

For Import mode, the status of the load displays as follows:

Evaluating...Waiting for other queries...Creating connection in model...Loading data to model...xxx rows from <vertica_server>;<db_name>.Detecting relationships...

13. Alternatively, to filter, transform, and shape your data before Load, click Edit.

The Query Editor opens.



For information about the Query Editor, see Shape and Combine Data on the Power BIDesktop website.

When the load operation completes, the selected tables and columns appear on the right-handside of the window, under Fields.

Connecting to Vertica via a Generic ODBC ConnectionYou can connect from Power BI Desktop to Vertica using a generic ODBC connection. Power BIDesktop imports data from Vertica into its cache to use in reports. When you work with yourvisualizations, you may need to re-import the Vertica data to make sure you have an up-to-datedataset. Before you re-import the data, you need to clear the Power BI Desktop cache. To doso, select File > Options and Settings > Options > Global Data Load.

Install the Vertica ODBC Driver

Important

If you are using the new Vertica named connector you don’t need to install Vertica’sODBC driver. The new Vertica connector bundles the appropriate driver.

Before you can connect Power BI Desktop to Vertica using the generic ODBC connection, youmust download and install the Vertica ODBC driver and create a 64-bit Data Source Name(DSN).

Follow these steps to install the driver:

1. Go to the Vertica Client Drivers page.

2. Download the version of the Vertica ODBC client package that is compatible with thearchitecture of your operating system.

Note

The Vertica ODBC driver is now both forward and backward compatible. The ODBCdriver was made backwards compatible starting in Vertica 8.1.1 to enableconnection to most Vertica server versions. For more information about client andserver compatibility, see Client Driver and Server Version Compatibility in theVertica documentation.

3. Based on the client driver package you downloaded, follow the installation instructions inthe Vertica documentation.

The client driver package provides several drivers, but you should install only the ODBCdriver.

Create a DSNPower BI is available as a 64-bit and 32-bit application. If you download and install the 32-bitdriver, create a 32-bit DSN. If you download and install the 64-bit driver, create a 64-bit DSN.

After you have installed the ODBC driver, you need to set up a DSN and configure the ReportUnicode columns as char setting.



https://docs.microsoft.com/en-us/power-bi/desktop-shape-and-combine-data

https://vertica.com/vertica-client-drivers/

https://www.vertica.com/docs/latest/HTML/index.htm#Authoring/ConnectingToVertica/ClientDriverMisc/ClientDriverAndServerVersionCompatibility.htm

https://www.vertica.com/docs/9.1.x/HTML/index.htm#Authoring/ConnectingToVertica/InstallingDrivers/Windows/InstallingTheClientDriversOnWindows.htm

To set up the DSN, follow the instructions in Setting Up an ODBC DSN in the Verticadocumentation.

To configure the Report Unicode columns as char setting, follow these steps:

1. Select Start > Control Panel > Administrative Tools > Data Sources (ODBC).

2. Click System DSN.

3. Select your DSN and click Configure.

4. Click the Client Settings tab.

5. Select Report Unicode columns as char. This setting tells the ODBC driver to tell Power BIDesktop that Vertica uses the ODBC CHAR data type.

Create the Connection1. To open Power BI Desktop, select Start > Microsoft Power BI Desktop > Power BI

Desktop.

2. In the initial window, on the Home tab, click Get Data.

3. In the Get Data window, click Other.

4. In the list of available connections, select ODBC and click Connect.

5. In the From ODBC window, select the DSN you created as described earlier in thisdocument.

The database you designated in your DSN appears in the left-hand column of the



https://www.vertica.com/docs/latest/HTML/index.htm#Authoring/ConnectingToVertica/ClientODBC/SettingUpADSN.htm

Navigator window.

6. Specify the data to import by writing a query or selecting tables.

Import Data from Vertica into Power BI DesktopIn the From ODBC window, after you have entered your DSN, you have two options forimporting data:

l Supply a Query to Import Data

Choose this option if you are working with very large tables. By providing a query, you canreduce the number of rows and columns that you are importing.

To enter a query, select Advanced options.

l Select Tables to Import

To select the tables from which you want to import data, click OK.

Supply a Query to Import Data1. In the From ODBC window, click Advanced options.

2. Under SQL statement (optional), enter a query. The example in this section uses thefollowing query to retrieve data from the inventory_fact table in VMart:

--Inventory fact table - analysisSELECT

date,full_date_description,day_of_week,calendar_month_name,calendar_month_number_in_year,calendar_year_month,calendar_quarter,calendar_year_quarter,calendar_year,

public.product_dimension.product_key || public.product_dimension.product_version as product_version_key,

product_description,sku_number,category_description,department_description,package_type_description,package_size,fat_content,diet_type,warehouse_name,



warehouse_city,warehouse_state,warehouse_region,qty_in_stock

FROMpublic.inventory_fact INNER JOIN public.date_dimensionON inventory_fact.date_key = date_dimension.date_keyINNER JOIN public.warehouse_dimensionON inventory_fact.warehouse_key = warehouse_

dimension.warehouse_keyINNER JOIN public.product_dimensionON inventory_fact.product_key = product_dimension.product_keyAND inventory_fact.product_version = product_dimension.product_

versionWHERE

public.date_dimension.date >= (DATE '2003-01-01') ANDpublic.date_dimension.date <= (DATE '2017-12-31') ANDdiscontinued_flag = 0;

The From ODBC window looks like this:

3. To continue with the import, click OK.

Power BI Desktop displays the query results:



4. To import the data from the selected tables, click Load.

The status of the load displays as follows:

Evaluating...Waiting for other queries...Creating connection in model...Loading data to model...Waiting for dsn=<my_dsn>.xxx rows from dsn=<my_dsn>.Detecting relationships...

5. Alternatively, to filter, transform, and shape your data before importing, click Edit.


For information about the Query Editor, see Shape and Combine Data on the PowerBI Desktop website.

When the load operation completes, the selected tables and columns appear on the right-handside of your window, under Fields.

Select Tables to Import1. In the From ODBC window, do not enter a query. Just click OK.

2. In the Access a Data Source using an ODBC driver window, type your database user nameand password if required.

3. Click Connect.

4. In the Navigator window, expand the database name to list the schemas.

5. Expand the schemas you are interested in and check the tables you want to import foryour analysis. In this example, you connect to the VMart example database and, from the



https://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-common-query-tasks#shape-and-combine-data

Public schema, import the inventory_fact table and its related dimension tables.

l date_dimension on date_key

l product_dimension on (product_key and product_version)

l warehouse_dimension on warehouse_key

The following image shows how to select the tables:

6. To import the data from the selected tables, click Load. The load status displays as follows:

Evaluating...Waiting for other queries...Creating connection in model...Loading data to model...Waiting for dsn=<my_dsn>.xxx rows from dsn=<my_dsn>.Detecting relationships...

7. Alternatively, to filter, transform, and shape your data before importing, click Edit.




For information about the Query Editor, see Shape and Combine Data on the PowerBI Desktop website

When the load operation completes, the selected tables and columns appear on the right-handside of the window, under Fields.

Connect to Vertica from Power BI ServiceAfter you design your reports in Power BI Desktop, you can choose to publish those reports toPower BI Service in order to share them with other users. You access Vertica from Power BIService when you open a published report. Power BI Service connects using a DirectQueryconnection via the on-premises data gateway.

Important

In order to refresh datasets based on Vertica data in the Power BI Service, you need toupgrade to the October 2018 release of the on-premises data gateway.

The October release enhancements do not involve the Power BI Report Server. SeePower BI Overview for information about the Power BI Report Server.

The Power BI On-Premises Data GatewayYou must install and configure the October 2018 release of the on-premises data gateway inorder to:

l Publish a DirectQuery-based report from Power BI Desktop to Power BI Service.

l Connect live to your Vertica database from Power BI Service.

Download and Install the On-Premises Data Gateway1. Download the on-premises data gateway from the Power BI downloads website.

2. Double click the downloaded file and follow the prompts for installation.

Note

You should install the gateway on a machine that is always up and running and thatPower BI users can access at all times.



https://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-common-query-tasks#shape-and-combine-data


Configure the On-Premises Data GatewayAfter installation, you must configure the gateway from Power BI Service so that it works withyour Vertica connection:

1. Log in to your Power BI Service account.

2. In the gear menu on the top right-hand side of the Power BI Service screen, clickManagegateways.



3. In the GATEWAY CLUSTERS screen, select from the list of gateways the name of thegateway you installed as described in Download and Install the On-Premises Data Gatewayin this document.

4. Verify that your gateway is up and running and displays as Online.

5. On the same screen, click ADD DATA SOURCE.

6. In the new screen, type a name for your Vertica data source in the Data Source Namebox, and select Vertica from the Data Source type list.



7. Type your Vertica database connection information, including Server, Database, Usernameand Password.



8. Click Add.

The connection should display as successful.



9. Repeat steps 5 to 8 for every unique Vertica connection you want to access from Power BIService.

Once you have configured your gateway as well as the Vertica connection that theDirectQuery-based report is using, you should be able to publish your report. You will see thefollowing message that confirms that publishing to Power BI Service was successful:



Lastly, you are now able to open the published report from Power BI Service (SaaS):

Known Issues and Work-AroundsReview these known issues and their workarounds when connecting to Vertica using PowerBI Desktop.

Publishing to Power BI Service fails using DirectQueryIssue #1: You may see the following error when publishing a DirectQuery-based report fromPower BI Desktop to Power BI Service if the on-premises data gateway is not installed:

Solution: Power BI Server requires the October 2018 release of the on-premises data gatewayin order to connect LIVE to Vertica. Install and configure the October 2018 release of the on-premises data gateway from: https://powerbi.microsoft.com/en-us/downloads/

Issue #2: You may see the following error when publishing to Power BI Service if the on-premises data gateway hasn’t been configured correctly inside Power BI Service:




Solution: After installing the on-premises data gateway, make sure to configure the gatewayin order to connect to Vertica using DirectQuery. The configuration should be performed inPower BI service > Manage gateways. To configure the on-premises data gateway in Power BIService, follow the instructions in Configure the On-Premises Data Gateway in this document.

DirectQuery doesn’t support DAX functions/transformationsIssue: DAX functions are not supported when working in DirectQuery mode. You may see anerror like the following:



Solution: If you require DAX functions to model the data and build the reports, try performingthe calculations and transformations in Vertica, for example, by using live aggregateprojections, as described in Use Live Aggregate Projections. Push down the execution of thesecalculations to the database, and use Power BI to create the visualizations only.

String Data Too Big ErrorIssue: You may see the following error when trying to load data from some Vertica data types:

Details: "ODBC: SUCCESS_WITH_INFO [01004] [Vertica][ODBC](10160) String data right truncation: String data is too big for theoutput data buffer and has been truncated.

Solution 1: Configure the Report Unicode columns as char setting as follows:

1. Select Start > Control Panel > Administrative Tools > Data Sources (ODBC).

2. Click System DSN.

3. Select your DSN and click Configure.

4. Click the Client Settings tab.

5. Select Report Unicode columns as char. This setting tells the ODBC driver to tell Power BI



https://www.vertica.com/docs/latest/HTML/Content/Authoring/AnalyzingData/AggregatedData/LiveAggregateProjections.htm

Desktop that Vertica uses the ODBC CHAR data type.

Solution 2: Upgrade to Power BI Desktop 2.35 or later.

Data Type LimitationsPower BI Desktop does not support all data types supported by Vertica. Power BI Desktop doesnot load unsupported data types into the cache or display the data in dashboards. The followingis a list of known limitations for data types:

l BINARY, VARBINARY, LONG VARBINARY data types are not supported.

l Long string values are truncated to 32766 characters.

l Milliseconds and time zone offset are not displayed for TIMETZ and TIMESTAMPTZ datatypes.

l INTERVAL YEAR TO MONTH, INTERVAL HOUR TO SECOND, INTERVAL HOUR TO MINUTE,and INTERVAL DAY TO SECONDS data types are not supported.

l Maximum number of digits in numeric values is 15.

l Some numeric values are displayed in scientific notation, for example:

-79228162514264337593543950335 is displayed as-7.92281625142643E+28



7922816251426433759354395.0335 is displayed as 7.92281625142643E+24

l Digits to the right of the decimal point are truncated, for example:

7.9228162514264337593543950335 is displayed as 7.92281625142643

79228162514264.337593543950335 is displayed as 79228162514264.3

l Some decimal values are rounded, for example:

–99999999999999.9999 to –100000000000000

l Large integer values are not supported. Maximum number of digits is 15. This errormessage is presented when loading very large integer values:

For information about Vertica data types, see SQL Data Types in the Vertica documentation.

For information about data types in Power BI Desktop, see Data types in Power BI Desktop.

For More Informationl Power BI

l Vertica Community Edition

l Vertica User Community

l Vertica Documentation



https://www.vertica.com/docs/latest/HTML/index.htm#Authoring/SQLReferenceManual/DataTypes/SQLDataTypes.htm

https://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-data-types/

https://powerbi.microsoft.com/en-us/

https://www.vertica.com/register/

https://www.vertica.com/big-data-analytics-community-content/

http://www.vertica.com/docs/latest/HTML/index.htm

VerticaKnowledgeBaseArticle VerticaIntegrationwith ... · Vertica Integration with Microsoft Power...

Documents

Transcript of VerticaKnowledgeBaseArticle VerticaIntegrationwith ... · Vertica Integration with Microsoft Power...