Tulsa Techfest 2016 : Pragmatic Governace by Scott Mitchell

Tulsa TechFest 2016 | Fri, Aug 5th, 2016 | OSU - Tulsa | 70+ Speakers, 20+ Tracks & 85+ Sessions!

Please help us!Thank our Sponsors:


Low Cost Governance with Microsoft Online ServicesLOW COST, RICH CAPABIL ITY & RESOURCE INVESTED

BY SCOTT MITCHELL


Session Objectives and Takeaways• Demonstrate how to use automation for applying IT/Information

Governance• Discover the features and capabilities of Azure Automation• Illustrate the power of data driven automation using XML & PowerShell• Perform CRUD operations on SharePoint Data using the REST API• Give a working demo!

And… My main objective is to• Inspire you to create solutions based on these ideas


• Training programs are ineffective at driving policy compliance. People do not remember and abide by policies without continued awareness updates and retraining• There is no immediate feedback when policies are inadvertently

violated• Tools that monitor and include end user communication and

awareness features are expensive and require more IT energy to operate• Tools that apply controls are not flexible when exceptions are

necessary and systems become brittle and hard to extend

Current State Problem Statements


Test Policy for Demonstration Purposes

ESTABLISHED POLICYPROJECT SITES ARE NOT TO HAVE SUBSITESPolicy Definition

Per PMO Office

Project sites should not be structurally modified such as renaming the status list, document library or adding subsites.Project sites have end user training and processes paired with the site itself that break down if process related or project related material are buried in subsites.

Policy Link https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?k=project%20site%20integrity

Exception Notice

Site Owners are notified of noncompliance

Remediation Steps

Site owner is tasked to place content in original locations and remove subsites.

• PMO claims that their business processes break down often when PMs alter site structures.

• IT does not have the ability to configure granular controls to manage this policy through permissions

• Training is costly and not effective• By the time problems are

detected rollups and planning metrics have already gone off track

https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?k=project%20site%20integrity

https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?k=project%20site%20integrity


CommunicationAutomation AssetsSharePoint Lists

InformationSourceData [List]-SourceMetadata-SourceMetrics

Policy Rules [List]-Expressions

($sourcedata.variable –operator ‘value’)

Owner Mailbox-Policy Exception Notices

-Policy Title-SourceLink

Check-SitePolicies [RunBook]

> For Each Site in SiteCatalog> Process All Site PolicyExpressions> Send Exception Message

Update-SourceData [RunBook]

For All Sites Create/Update Site Details Invoke Check-SitePolicies

Schedule : TwiceDaily_1100Hrs

Policy Log Mailbox-Copy of all notices

Solution OverviewSolution Overview

IT Department/Policies Pages -Policy Details

-search?policypages=policy title


• Add SharePoint PowerShell Support to Azure Automation Runbook• Download the SharePoint Online Management Shell• https://www.microsoft.com/en-us/download/details.aspx?id=35588• Next Place it in a ZIP file and upload to your runbook Assets.• All Client and Runtime Libraries are included• All Assemblies are loaded automatically. So there is no need to add the types.

Setup and Configuration

https://www.microsoft.com/en-us/download/details.aspx?id=35588


• Runbook Assets are available at runtime including uploaded modules• Initializing for runtime helps so that you can run the same scripts

locally• Schedules, Logs, Source Control are all available and easy to use

Azure Automation

$spurl = 'https://stmtrial01-admin.sharepoint.com'$spsite = 'https://stmtrial01.sharepoint.com'if($env:SESSIONNAME -eq 'Console'){

$creds = Get-Credential -UserName '[email protected]' -Message 'Login'Import-Module Microsoft.Online.SharePoint.PowerShell$path = (Get-Module Microsoft.Online.SharePoint.PowerShell).ModuleBase#[Void][System.Reflection.Assembly]::LoadFrom("$Path/Microsoft.SharePoint.Client.dll")#[Void][System.Reflection.Assembly]::LoadFrom("$Path/Microsoft.SharePoint.Client.Runtime.dll")connect-sposervice -url $spurl -credential $creds$credential = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($creds.UserName, $creds.Password)}

else{$creds = get-AutomationPSCredential -name 'stmtrial'connect-sposervice -url $spurl -credential $creds$credential = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($creds.username, $creds.password)

}


What can runbooks automate?Runbooks in Azure Automation are based on Windows PowerShell or Windows PowerShell Workflow, so they do anything that PowerShell can do. If an application or service has an API, then a runbook can work with it. If you have a PowerShell module for the application, then you can load that module into Azure Automation and include those cmdlets in your runbook. Azure Automation runbooks run in the Azure cloud and can access any cloud resources or external resources that can be accessed from the cloud. Using Hybrid Runbook Worker, runbooks can run in your local data center to manage local resources.

From - https://azure.microsoft.com/en-us/documentation/articles/automation-intro/

Azure Automation

https://azure.microsoft.com/en-us/documentation/articles/automation-hybrid-runbook-worker/

https://azure.microsoft.com/en-us/documentation/articles/automation-intro/


PowerShell XML Supportfunction Set-SPOListItem <#<listitem rootSiteCollection='https://stmtrial01.sharepoint.com' listName='SiteStatus' keyField='Title'> <fields> <field name='Title' type='TEXT'>Hello789</field> <field name='Owner' type='TEXT'>Momma Cass</field> <field name='IssueStatus' type='CHOICE'>Resolved</field> <field name='DateCompleted' type='DATE'>8/1/2016</field> </fields></listitem>#># POSTING LIST DATA TO SHAREPOINT #$listName = [string]$ListData.listitem.listName $list = Invoke-SPORestMethod -Url "$siteUrl/_api/web/lists/getbytitle('$listName')" -Method Get -Credentials $Credential $Uri = $list.__metadata.uri $ListType = $list.ListItemEntityTypeFullName $keyFieldName = [string]$ListData.listitem.keyField $keyFieldValue = [string]$ListData.SelectSingleNode("//field[@name='$keyFieldName']").innerText $keyFieldType = [string]$ListData.SelectSingleNode("//field[@name='$keyFieldName']").type $CamlQuery = [string]"{{ 'query' : {{'__metadata': {{ 'type': 'SP.CamlQuery' }}, `"ViewXml`": `"<View><Query><Where><Contains><FieldRef Name='{0}'/><Value Type='{1}'>{2}</Value></Contains></Where></Query></View>`" }} }}" -f $keyFieldName, $keyFieldType, $keyFieldValue $items = Invoke-SPORestMethod -Url "$uri/getitems" -Method Post -Credentials $Credential -RequestDigest $digest -Metadata $CamlQuery -XHTTPMethod Post if ($items.results.count -eq 1) { #Update Items with new values $itemUri = $items.results[0].__metadata.uri $ListXMLValues = $ListData.SelectNodes("//field") $jsonListItems = [string]"{'__metadata': { 'type': '$listType' }, " foreach ($item in $ListXMLValues) { $fieldname = [string]$item.name $fieldname = $fieldname.replace(' ','_x0020_') $jsonListItems += [string]"'{0}': '{1}'," -f $fieldname, $item.innerText } $jsonListItems += "}" $jsonListItems = $jsonListItems.Replace(',}','}') $result = Invoke-SPORestMethod -Url $itemUri -Method Post -Credentials $Credential -RequestDigest $digest -Metadata $jsonListItems -XHTTPMethod Merge -ETag '*'


PowerShell Mapping Fields and Insert/Updating Items $sitesdata = [System.Xml.XmlDocument]::new() $listitem = $sitesdata.CreateElement('listitem') $listitem.SetAttribute('rootsitecollection',$CatalogSite) $listitem.SetAttribute('listname',$CatalogListName) $listitem.SetAttribute('keyfield',$CatalogKey) [void]$sitesdata.AppendChild($listitem) $fields = $sitesdata.createelement('fields') [void]$sitesdata.SelectSingleNode("//listitem").AppendChild($fields)

foreach ($site in $sites) { #Makesure fields element is clear... $fields = $sitesdata.SelectSingleNode("//fields") $fields.RemoveAll() #Then add all relevant site fields... #Field names are case sensitive... Use matching case when mapping fields.

$field = $sitesdata.createelement('field') $field.SetAttribute('name','Title') $field.SetAttribute('type','TEXT') $field.InnerText = $site.url [void]$fields.AppendChild($field)

. . .

Set-SPOListItem -ListData $sitesdata -Credential $Credential }

• Programmatically build an xml representation of a list item.• This xml is passed to the

set-spolistitem cmdlet which makes it so.


Dynamic Script Evaluating Policy Rules

foreach($sitestatus in $SiteStatus.results.GetEnumerator()){ $Notices = @() #Process all policy rules foreach($policy in $Policies.results.GetEnumerator()){ $sb = [Scriptblock]::Create($policy.ExceptionExpression) $Exception = $sb.invoke() if($Exception){ $notice = New-Object System.Object $notice | Add-Member -MemberType NoteProperty -Name 'Owner' -Value $SiteStatus.owner $notice | Add-Member -MemberType NoteProperty -Name 'Source' -Value $SiteStatus.Title $notice | Add-Member -MemberType NoteProperty -Name 'Policy' -Value $Policy.Title $notices += $notice } } #End For Each Policy if($notices){ $body="" foreach ($notice in $notices){ $PolicyTitle = $notice.Policy $Owner = $notice.Owner $sourceref = $Notice.Source $fromAddress = '[email protected]' $subject = 'Automated Policy Exception Notice' $body += @"

. . . "@ $creds = get-AutomationPSCredential -name 'stmtrial' send-mailmessage -to $owner -from $FromAddress -subject $subject -body $body -usessl -port 587 -smtpserver 'smtp.office365.com' -credential $Creds -BodyAsHtml } } } #End For Each Site Status

#TODO : tokenize and validate expressions rather than accept any value from the outside. This is just a demo! I know the security guys are heating up right now because I just accepted foreign values in my code as dynamic script.


Loose Coupling – Search Based Policy Links $body += @"<h1>Automated Policy Audit Notice</h1>Dear Information Owner,<p/>Please review this notice and take corrective actions in an effort to comply with established electronic data policies.<h2>Policy Detail</h2><blockquote>Policy Name: $policyTitle</blockquote><blockquote><a href='https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?u=https%3A%2F%2Fstmtrial01%2Esharepoint%2Ecom%2Fsites%2Fit&k=$policyTitle'>Policy Details</a></blockquote><h3>Information Source</h3><blockquote><a href='$sourceref'>Information Source Link</a></blockquote><h3>Information Owner</h3><blockquote>$Owner</blockquote>"@

'https://stmtrial01.sharepoint.com/sites/it/_layouts/15/osssearchresults.aspx?u=https%3A%2F%2Fstmtrial01%2Esharepoint%2Ecom%2Fsites%2Fit&k=$policyTitle


Solution Walkthrough and Demo

Azure Automation Jobhttps://portal.azure.com/

SharePoint Sitehttps://stmtrial01.sharepoint.com

Solution Demo

https://portal.azure.com/

https://stmtrial01.sharepoint.com/


Add SiteSharing Policy Exception to the Policy listhttps://stmtrial01.sharepoint.com/Lists/Policies Restricted Intranet Site Sharing($sitestatus.sharing -ne 'Disabled') -AND ($sitestatus.Template -like 'BLANKINTERNET#0')

Rerun the Compliance Script and notice the new email sent to:[email protected]://outlook.office.com

Add a New Policy Expression

https://stmtrial01.sharepoint.com/Lists/Policies

mailto:[email protected]

https://outlook.office.com/


Session Objectives and TakeawaysSession Recap• Demonstrated a practical automated audit solution for modern IT• Discovered the features and capabilities of Azure Automation• Illustrated the power of data driven automation using XML & PowerShell• Performed CRU operations on SharePoint Data using the REST API

And… now you are thinking…

I can replace all human contact with a Site and an Automation Job


Additional Resources

Free Microsoft eBookAzure AutomationBy Michael McKeown

Don’t forget the Module Gallery. You may make requests.

Office Dev Centerhttps://msdn.microsoft.com/en-us/library/office/dn531433.aspx

https://msdn.microsoft.com/en-us/library/office/dn531433.aspx

https://msdn.microsoft.com/en-us/library/office/dn531433.aspx


Please Complete An Evaluation FormYour input is important!

You can access Evaluation Forms at:

http://TulsaTechFest.com

Fill them out!

You can win additional prizes!

Like a $50 Best Buy Gift Card!!

Winner drawn – Midnight, Sun Aug 7th!

http://tulsatechfest.com/

Tulsa Techfest 2016 : Pragmatic Governace by Scott Mitchell

Leadership & Management

Transcript of Tulsa Techfest 2016 : Pragmatic Governace by Scott Mitchell