Implementing Agile Data Governance

IMPLEMENTING AGILE DATA GOVERNANCE

Tami FlowersDirector, Governance SolutionsMetaGovernance Solutions, LLC

Data Governance Winter Conference – Nov 2016

WHO AM I? Tami Flowers

Director, Governance Solutions at MetaGovernance PMI-PMP, PMI-ACP, and Certified Scrum Master Agile Coach

MetaGovernance Solutions LLC. All Rights Reserved. 2016

Developer Project Manager

Enterprise Architect

Agile Coach

Data Governan

ce Consultan

t

TODAY Commonalities of all Agile/Lean flavors and how they apply to

Data Governance. How to break down and execute the Data Governance roadmap

via projects. How to integrate Data Governance with project teams. How to write governance stories, including real world examples.

A few minutes on the topic An “experience” Debrief/Reflection


EXPERIENCE: ICE BREAKER Write your answer to the following questions on sticky

notes. One answer per note. As many answers as you wish. What are you hoping to learn today? What specific questions are you hoping get answered today?

Discuss amongst your team. Group like answers together.


DATA GOVERNANCE FRAMEWORK

Data Governance OrganizationData Management FunctionsProjects


Agile Data Governance

DATA GOVERNANCE ORGANIZATION FRAMEWORK


Executives

Business & Technical Members

Stewards/analysts/architects/ administrators/engineers

Consumers

Data Governance Council (DGC)

Data Governance Steering Committee

(DGSC)

Data Stewardship Teams (virtual)

• Establish & train DGC members• Agree on Data Governance roadmap• Determine members & empower DGSC• Define Data Governance policy• Meet on recurring basis

• Receive training and education• Execute DG roadmap• Define DG communication plan• Select & empower stewardship

teams• Define roles and responsibilities• Define charter• Meet on recurring basis

• Technical and Business Members• Receive training and education• Execute DG roadmap

• Business Members • Receive training and

educationEnd Users

DATA GOVERNANCE ORGANIZATION FRAMEWORK ACTIONABLE DATA GOVERNANCE


Executives

Business & Technical Members

Stewards/analysts/architects/ administrators/engineers

Consumers

Data Governance Council (DGC)

Data Governance Steering Committee

(DGSC)

Data Stewardship Teams (virtual)

• Lead and promote program• Empower DGSC• Arbitrate data related issues, as needed

• Execute DG communication plan• Define data management policies,

procedures and standards• Approve stakeholders, and sources and use

of data per subject area• Approve and oversee corrective actions

• Technical and Business Members• Implement policies, procedures,

standards and controls• Define stakeholders, and sources

and use of data per subject area• Identify and report data issues• Corrective action input & execution• Corrective action status reporting

• Business Members that consume and use data

• Identify and report data issues

End Users

THE AGILE MANIFESTOWe are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over Processes and toolsWorking software over Comprehensive documentationCustomer collaboration over Contract negotiationResponding to change over Following a plan

That is, while there is value in the items on the right, we value the items on the left more.


In February 2001, 17 software developers met at the Snowbird resort in Utah to discuss lightweight development methods. They published the Manifesto for Agile Software Development.

AGILE PRINCIPLES Our highest priority is to satisfy the customer

through early and continuous deliveryof valuable software governance and governance processes.

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Deliver working software governance frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects governance around motivated individuals. Give them the environment and support they need, and trust them to get the job done.


AGILE PRINCIPLES The most efficient and effective method of

conveying information to and within a development team is face-to-face conversation.

Working software (and sustainable) governance is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity--the art of maximizing the amount of work not done--is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.


THE MANY FLAVORS OF LEAN/AGILE


Eliminate Waste

Empower the Team

Deliver Fast

Optimize the

WholeBuild

Quality In

Defer Decisions

Amplify Learning

COMMONALITIES OF ALL AGILE/LEAN FLAVORS

Capture the work Make it visible Minimally Viable Product

(MVP) Engage the

owner/customer/ stakeholders

Have owner prioritize the work

Work on the highest priority work first

Iterate Continually retrospect and

improve


EXPERIENCE: DATA GOVERNANCE TO LEAN PRINCIPLES

Agile Practice/Lean Principle

Eliminate Waste

Empower the Team

Deliver Fast

Optimize the Whole

Build Quality in

Defer Decisions

Amplify Learning

Teams make decisions

Just in time planning

Retrospectives

Frequent iterations

Validate/approve as we goInvolve the business stakeholders MetaGovernance Solutions LLC. All Rights Reserved. 2016

REALITY CHECK Agile isn’t a silver bullet that will magically fix

everything New, unexpected challenges will occur

Teamwork Soft skills Growing pains Will surface ineffective processes Will surface under-performing team members


REALITY CHECK Data Governance isn’t a silver bullet that will be easy to

quickly implement Must include the business Must be added to existing processes to be successful Lots of training and education is needed Executive support is critical Data governance framework must be in place Will uncover data issues that some don’t want uncovered


DATA GOVERNANCE ROADMAP


Data Strategy

Data Governance Roadmap

Operational Data

Governance

EXPERIENCE: BREAK DOWN DATA GOVERNANCE ROADMAP Identify components that should go into a Data

Governance Roadmap Which of these do you need for any and every project?


DATA GOVERNANCE ROADMAP -> OPERATIONAL GOVERNANCE WITHIN PROJECTS What types of projects need governance?


Operational Data Governance (recurring)EDW Content Specific Data Governance - Identify subject area(s)- Determine DG stakeholder roles- Associate and define business

attributes- Define system of record- Populate governance metadata- Define DG controls for compliance and

operational efficiency- Define DQ standards and measures- Define DQ metrics- Define other needed controls and

standards- Implement automated DG controls- Implement DQ standards - Implement automated metric capture- Create project-related reports and

dashboards- Create DQ reports and dashboards- Incorporate DG control results,

metrics and DQ measures into DG framework and workflow

- Identify, document, research data issues and recommend corrective actions

- Implement approved corrective actions

Data Governance Roadmap (SAMPLE)Q4

2016Q1

2017Q2

2017Q3

2017Q4

2017


DATA GOVERNANCE ARTIFACTS WITHIN A PROJECT Stories

As a <role> I want <what> so that <business value> If not a story, include data governance as acceptance

criteria in stories As an accountant, I want current balances for loans, so that I

can complete daily reporting All data elements are defined in the business dictionary Data profiling of the balances occurs when loading into the EDW An automated control is added so the current balance is

automatically reconciled between the G/L and Loan System


EXPERIENCE: ADDING GOVERNANCE TO A PROJECT Identify what is needed to add governance to a project. What 2-3 items are the highest priority?

Project: bring new data source in to a data warehouse Needs: Ensure all registered stakeholders and their data

governance roles are known. Identify and define business attributes. Profile data for potential issues.


GROUP EXPERIENCE: WHAT STORIES ARE NEEDED IF WE ARE DELIVERING A NEW BI REPORT? Identify stakeholders Identify source(s) of data for report Ensure business definitions exist Is there protected data on the report? What is the retention of the report? How frequently should the report be delivered? What calculations are required? Define SLAs Id other key metadata around report – use, purpose, audience What is the quality of the source(s) – profiling, metadata of the source What security and access is needed?


GROUP EXPERIENCE: WHAT STORIES ARE NEEDED IF WE ARE BRINGING IN A NEW DATA SOURCE? Identify data quality dimensions and thresholds Identify archiving and data retention needs Classify the data for visibility and security Identify data stewards Establish profiling and cleansing rules Establish SLAs with the provider Identify legal aspects, regulatory impacts Identify restrictions on the data, permissible use Evaluate risk involved with data Determine requirements to match to master or reference data; identity attributes Determine upwards and downward impacts to business processes Configuration management; hardware/software needed, documentation Define metadata, definitions, associations.


WAYS AGILE CAN HELP Increased communication and visibility Increased teamwork More frequent delivery of business value Implement governance on high priority items first, show

success, move on…continuous improvement


CONTINUOUS IMPROVEMENT Use retrospectives to make your process better It’s ok to add just 1 or 2 new things at a time Get help if you need it, lots of resources and experts are

out there Mentoring and coaching will be critical


LESSONS LEARNED There isn’t one definition of Agile; all companies tend to do it

differently. That’s ok. Make it what works best for your organization.

If you are setting up governance for the first time, get the framework in place first. Then determine low hanging fruit, such as data quality issues, and implement governance starting there.

Data Governance MUST involve the business; it can’t just be an IT focus.

A lot of training and education is needed for governance. Data Governance must be added to existing processes and

procedures in order to give it teeth. SDLC Architecture reviews


EXPERIENCE: TEMPERATURE READING Appreciations Puzzles Complaints with recommendations Hopes and wishes New Information


RETROSPECTIVES


At end of each sprint/iteration What went well – keep doing What didn’t go well – stop doing What could go better – change it

Entire team attends

EXPERIENCE: RETROSPECTIVE


What went well What didn’t go well What could go better

GOOD RESOURCES FOR AGILE AND TEAM BUILDING Mike Cohn – MountainGoatSoftware.com PMI-ACP Exam Prep – Mike Griffiths Johanna Rothman books: http://www.jrothman.com/books/ The Human Side of Agile, Gil Broza Agile Retrospectives: Making Good Teams Great, Esther Derby and Diana Larsen Crucial Conversations, Kerry Patterson Crucial Confrontations, Kerry Patterson Influencer The Deadline: A Novel about Project Management, Tom DeMarco Teamwork is an Individual Skill, Christopher Avery Behind Closed Doors: Secrets of Great Management, Johanna Rothman and

Esther Derby DGIQ presentations: Roberta, Maggie from GM


http://www.jrothman.com/books/

ME Twitter: TamiLFlowers LinkedIn: Tami Flowers Email: [email protected] Slideshare: www.slideshare.net\tamiflowers www.metagovernance.com (blog) Blog for DataVersity

Thanks!


http://www.slideshare.net/tamiflowers

http://www.metagovernance.com/

Implementing Agile Data Governance

Technology

Transcript of Implementing Agile Data Governance