BI Markup Language - BI to the next level

#sqlsatParma

#sqlsat355November 22nd, 2014

BI Markup Language - BI to the next level

Davide Mauri

SolidQ – Founder & Mentor

@mauridb

www.solidq.com

[email protected]

#sqlsatParma


Sponsors

#sqlsatParma


Organizers

#sqlsatParma


Davide Mauri

20 Years of experience on the SQL Server Platform Specialized in Data Solution Architecture, Database Design,

Performance Tuning, Business Intelligence

Projects, Consulting, Mentoring & Training Regular Speaker @ SQL Server events

Microsoft SQL Server MVP

President of UGISS (Italian SQL Server UG)

Mentor @ SolidQ

Find me here: Blog: http://sqlblog.com/blogs/davide_mauri/default.aspx

Twitter: @mauridb

http://sqlblog.com/blogs/davide_mauri/default.aspx

#sqlsatParma


Agenda

BI, DWH & ETL today

BIML – What’s that?

Tools

BIDS Helper

MIST

BIML Kick Off

Basic Concepts

Automation With “Embedded” C#

BIML & PowerShell

#sqlsatParma


Building a BI/DWH/ETL in 2014

Is still a (almost) manual process

A *lot* of repetitive low-value work

No (or very few) standard tools available

#sqlsatParma


How it should be

Semi-automatic process

“develop by intent”

Define the mapping logic from a semantic perspective

Source to Dimensions / Measures

(Metadata anyone?)

Design the model and let the tool build it for you

CREATE DIMENSION Customer

FROM SourceCustomerTable

MAP USING CustomerMetadata

ALTER DIMENSION Customers

ADD ATTRIBUTE LoyaltyLevel

AS TYPE 1

CREATE FACT Orders

FROM SourceOrdersTable

MAP USING OrdersMetadata

ALTER FACT Orders

ADD DIMENSION Customer

#sqlsatParma


The perfect BI process & architecture

Iterative!

#sqlsatParma


Design Pattern

“A general reusable solution

to a commonly occurring

problem within a given

context”

#sqlsatParma


No Monkey Work!

Let the people think and let

the machines do the

«monkey» work.

#sqlsatParma



Business Intelligence Markup Language

XML-Based

BI Object-Generator

SQL Server

Integration Services

Analysis Services

#sqlsatParma



Allows a declarative approach to define BI objects

Favor automation

Human-readable

Compatible with SCC system

Allows merge of files

Born from “Project Vulcan”

Still available on CodePlex

#sqlsatParma



Once objects are generated, BIML is not needed

anymore

No additional runtime/components required

Support for SQL Server 2008 & 2012 objects

2014 is coming very soon

Anyway generated packages are automatically converted

into 2014 packages by Visual Studio 2013, without problems

#sqlsatParma



BIMLScript is based on the T4 templating engine.

Allows the generation of BIML using

C#

VB.NET

BIML + BIMLScript = HTML + ASP Classic

#sqlsatParma


Tools

BIDS Helper

http://bidshelper.codeplex.com/

Visual Studio Plugin

Free

Source Code Available

Not all BIML features are supported

“Limited” to SSIS generation only

http://bidshelper.codeplex.com/

#sqlsatParma


Tools

MIST

Full-Featured BIML IDE

Buy or Rent

Made by BIML creator: Varigence

#sqlsatParma


DEMO

First Contact with BIML

#sqlsatParma


Basic Concepts

A single BIML file can generate more than one SSIS

Package

All SSIS object and related properties are exposed via

BIML elements

Additional specific attribute: ConstraintMode

Linear or Parallel

#sqlsatParma


Basic Concepts

Parallel Constraint Mode

“Inputs” connected to “Outputs”

SSIS Variables: “dot” notation

#sqlsatParma


Basic Concepts

Control Flow and Data Flow 100% supported

In Data Flow it’s mandatory to define input and outputs

Since they aren’t constraints, but data streams

#sqlsatParma


DEMO

BIML is your friend!

#sqlsatParma


Automation with BIMLScript

BIML Script (T4) Directives: <#@ e #>

#sqlsatParma



BIML Script Tags: <# e #>

Allows usage of C# / VB.Net within BIML

The “=” symbol allows “in-line” code injection

#sqlsatParma



Metadata is needed in order to define what to generate

Metadata can be extracted by

BIML Engine (Hadron)

T-SQL Queries

#sqlsatParma



Metadata via BIML Engine (Hadron)

#sqlsatParma



Metadata via SQL Query

#sqlsatParma


DEMO

BIMLScript RULES!

#sqlsatParma


BIML & PowerShell

What if you don’t like C#...

Or if BIMLScript looks to complex?

Especially without MIST?

One option is to generate BIML using PowerShell

Is just XML after all….

#sqlsatParma


DEMO

BIML & PowerShell

#sqlsatParma


Conclusions

Use BIML!!!!

It helps to

Make better use of your time

Create your own ETL patterns

http://www.slideshare.net/davidemauri/automating-dwh-patterns-

through-metadata

Support Refactoring & Enables Agility

Favor use of metadata

http://www.slideshare.net/davidemauri/automating-dwh-patterns-through-metadata

#sqlsatParma


References

BIML Script

http://bimlscript.com/

Stairway to BIML

http://www.sqlservercentral.com/stairway/100550/

Automating DWH Patterns Through Metadata

http://www.slideshare.net/davidemauri/automating-dwh-

patterns-through-metadata

http://bimlscript.com/

http://www.sqlservercentral.com/stairway/100550/

http://www.slideshare.net/davidemauri/automating-dwh-patterns-through-metadata

#sqlsatParma


Q&A

Questions?

#sqlsatParma


THANKS!

#sqlsatParma

#sqlsat355

BI Markup Language - BI to the next level

Data & Analytics

Transcript of BI Markup Language - BI to the next level