BI software drives value: The next wave of customers’ business strategies
BI Markup Language - BI to the next level
-
Upload
davide-mauri -
Category
Data & Analytics
-
view
463 -
download
1
description
Transcript of BI Markup Language - BI to the next level
![Page 1: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/1.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
BI Markup Language - BI to the next level
Davide Mauri
SolidQ – Founder & Mentor
@mauridb
www.solidq.com
![Page 2: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/2.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Sponsors
![Page 3: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/3.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Organizers
![Page 4: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/4.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Davide Mauri
20 Years of experience on the SQL Server Platform Specialized in Data Solution Architecture, Database Design,
Performance Tuning, Business Intelligence
Projects, Consulting, Mentoring & Training Regular Speaker @ SQL Server events
Microsoft SQL Server MVP
President of UGISS (Italian SQL Server UG)
Mentor @ SolidQ
Find me here: Blog: http://sqlblog.com/blogs/davide_mauri/default.aspx
Twitter: @mauridb
![Page 5: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/5.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Agenda
BI, DWH & ETL today
BIML – What’s that?
Tools
BIDS Helper
MIST
BIML Kick Off
Basic Concepts
Automation With “Embedded” C#
BIML & PowerShell
![Page 6: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/6.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Building a BI/DWH/ETL in 2014
Is still a (almost) manual process
A *lot* of repetitive low-value work
No (or very few) standard tools available
![Page 7: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/7.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
How it should be
Semi-automatic process
“develop by intent”
Define the mapping logic from a semantic perspective
Source to Dimensions / Measures
(Metadata anyone?)
Design the model and let the tool build it for you
CREATE DIMENSION Customer
FROM SourceCustomerTable
MAP USING CustomerMetadata
ALTER DIMENSION Customers
ADD ATTRIBUTE LoyaltyLevel
AS TYPE 1
CREATE FACT Orders
FROM SourceOrdersTable
MAP USING OrdersMetadata
ALTER FACT Orders
ADD DIMENSION Customer
![Page 8: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/8.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
The perfect BI process & architecture
Iterative!
![Page 9: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/9.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Design Pattern
“A general reusable solution
to a commonly occurring
problem within a given
context”
![Page 10: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/10.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
No Monkey Work!
Let the people think and let
the machines do the
«monkey» work.
![Page 11: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/11.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
Business Intelligence Markup Language
XML-Based
BI Object-Generator
SQL Server
Integration Services
Analysis Services
![Page 12: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/12.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
Allows a declarative approach to define BI objects
Favor automation
Human-readable
Compatible with SCC system
Allows merge of files
Born from “Project Vulcan”
Still available on CodePlex
![Page 13: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/13.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
Once objects are generated, BIML is not needed
anymore
No additional runtime/components required
Support for SQL Server 2008 & 2012 objects
2014 is coming very soon
Anyway generated packages are automatically converted
into 2014 packages by Visual Studio 2013, without problems
![Page 14: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/14.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
BIMLScript is based on the T4 templating engine.
Allows the generation of BIML using
C#
VB.NET
BIML + BIMLScript = HTML + ASP Classic
![Page 15: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/15.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Tools
BIDS Helper
http://bidshelper.codeplex.com/
Visual Studio Plugin
Free
Source Code Available
Not all BIML features are supported
“Limited” to SSIS generation only
![Page 16: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/16.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Tools
MIST
Full-Featured BIML IDE
Buy or Rent
Made by BIML creator: Varigence
![Page 17: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/17.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
First Contact with BIML
![Page 18: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/18.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Basic Concepts
A single BIML file can generate more than one SSIS
Package
All SSIS object and related properties are exposed via
BIML elements
Additional specific attribute: ConstraintMode
Linear or Parallel
![Page 19: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/19.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Basic Concepts
Parallel Constraint Mode
“Inputs” connected to “Outputs”
SSIS Variables: “dot” notation
![Page 20: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/20.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Basic Concepts
Control Flow and Data Flow 100% supported
In Data Flow it’s mandatory to define input and outputs
Since they aren’t constraints, but data streams
![Page 21: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/21.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
BIML is your friend!
![Page 22: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/22.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
BIML Script (T4) Directives: <#@ e #>
![Page 23: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/23.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
BIML Script Tags: <# e #>
Allows usage of C# / VB.Net within BIML
The “=” symbol allows “in-line” code injection
![Page 24: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/24.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
Metadata is needed in order to define what to generate
Metadata can be extracted by
BIML Engine (Hadron)
T-SQL Queries
![Page 25: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/25.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
Metadata via BIML Engine (Hadron)
![Page 26: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/26.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
Metadata via SQL Query
![Page 27: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/27.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
BIMLScript RULES!
![Page 28: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/28.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML & PowerShell
What if you don’t like C#...
Or if BIMLScript looks to complex?
Especially without MIST?
One option is to generate BIML using PowerShell
Is just XML after all….
![Page 29: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/29.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
BIML & PowerShell
![Page 30: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/30.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Conclusions
Use BIML!!!!
It helps to
Make better use of your time
Create your own ETL patterns
http://www.slideshare.net/davidemauri/automating-dwh-patterns-
through-metadata
Support Refactoring & Enables Agility
Favor use of metadata
![Page 31: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/31.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
References
BIML Script
http://bimlscript.com/
Stairway to BIML
http://www.sqlservercentral.com/stairway/100550/
Automating DWH Patterns Through Metadata
http://www.slideshare.net/davidemauri/automating-dwh-
patterns-through-metadata
![Page 32: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/32.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
Q&A
Questions?
![Page 33: BI Markup Language - BI to the next level](https://reader033.fdocuments.us/reader033/viewer/2022042602/559704fa1a28ab674f8b4866/html5/thumbnails/33.jpg)
#sqlsatParma
#sqlsat355November 22nd, 2014
THANKS!
#sqlsatParma
#sqlsat355