Release Management/SCMwith TFSScott Colestock

Presented by

About me -

• Consulting independently in the BizTalk 2004/2006 space for about 3 years

• Blog: www.traceofthought.net• Recently focused on VSTS/TFS

(speaking/writing/production rollouts)

Agenda

• Branching & Merging – Mechanics within TFS– Strategies

• How does Team Build help with SCM?• Carrying through to physical environments

– TFS Deployer

Software Configuration Management

• The process of identifying and selecting all the elements of a software system that create the correct and cohesive set - at defined points in time– Not unlike managing bill-of-materials for a

highly complex manufactured product

• Large topic – we’re focused (primarily) on branching & merging strategies…

• Poor SCM leads to development, testing, and operational inefficiencies

Branching Mechanics in TFS• TFS implements “path space branching” (vs.

“version space”)– Similar to subversion– CVS/others hide the branch…

• Branching in TFS is essentially like doing a “deep copy” of a folder on your local drive– But branch “parent” is preserved (where in the tree you

branched from)– Originating changeset is preserved– Ability to selectively merge parent changesets into the

branch

Branching Mechanics in TFS

• Path space branching means you see the branch folder directly in the Source Control Explorer – easy to navigate, retrieve, and establish permissions per branch

• TFS implements an immediate branching model, rather than deferred (i.e. branch-on-change)– More meta-data created at the point of branch

operation (though just one copy of underlying files until changed in branch)

– A bit more useable – contents of branch are clear

Branching Mechanics

• When creating a project, plan for future branches by placing initial check-in under a “Dev” or “Main” folder – not at your Team Project’s source root

• Branch operation asks you to specify a “branch target” – a location in the source control hierarchy where the branch should live

Branching Mechanics

• Most of the time, this will be a peer – e.g. $/OrderSystem/QA (or whatever the purpose of the branch is)

• Keep clear on the differences between branch parent/child relationships and the physical structure you observe on disk and in source control repository…

Branching Mechanics

• Branch operations result in a set of pending changes (of type “branch”) which must be checked in before branch is made real

• Branching can cross TFS projects as well– Useful for sharing code across

projects when you still desire to build/unit-test independently

Branching Thoughts

• Mental model: Just like workspaces provide isolation for a given developer, branches provide isolation for a group of developers

• Only branch if you have to, and at the point in time required– Branches can happen “after the fact” via

branch-from-label, branch-from-changeset, etc.– (Like at the point in time you need to create a

hotfix…)

Branching Thoughts• Always ask: Will the cost of merging conflicts in

“real time” be higher or lower than the cost of merging conflicts between branches?– Isolation can offer greater near-term productivity (more

parallelism)– Trade off is the risk of instability introduced by the

merge (which increases with time-between-merge)– Merges can often need to flow both ways (“reverse”

and “forward” integration)

• Don’t use branching to maintain different configurations of the same product. Do this at build time, packaging time, or deployment time, etc.

Simplest Branching Model: None

• Keep in mind:– TFS Team Builds are labeled automatically, or you can

apply your own label– You can branch-by-label at a later point in time if you need

to…– You can modify the contents of a label for small-scale

changes

• If your team moves forward together and you have a single release point (e.g. your web site) you may never need to branch– Do you need to actually do service releases/patches?

• Or do you just do another “point” release each month or so?

– Do people work “ahead” on items you don’t intend to release in your next drop?

(Label Detour)

Branch by Label

Label Thoughts• Labels allow you to apply a marker to the current (or past)

state of a file or folder• Usually used to denote a milestone or release• You can apply multiple labels to a version of a file or

folder…• Contents of a label can be edited after the fact (but no

version history of the label is maintained)• Team Build automatically assigns a label to the set of files

associated with each build that it creates.• Trivia: ‘Label scope’ is a path within source control -

underneath which no other labels with the same name can be created– When you use Source Control Explorer, scope = project root

Visualizing Labels

234

185

97

117

103

165

193

• The “contents” of a label is a set of changesets– The minimum number of

changesets needed to describe the state

• All that matters is the “aerial view”

• The fact that other changesets were involved in reaching the labeled state is irrelevant

• Label can be described by the content of the changesets “visible” from the top looking down… Label: 1.05 Release

Label Mechanics• In Source Control Explorer, right-click at appropriate

level and select “Label” to apply a label– To “latest”– Or by date– Or by changeset– (Or by label – Build “Dev Debug_20070319” is also Version 2.4)

• To find/delete/edit – use File-Source Control-Label-Find Label– Edit allows you to choose additional “participants” in the label or

later versions of existing participants

• Use http://shurl.org/tfskicks for advanced label features– Label comparison– Label changeset contents (union of all participant changesets)– Label work items

Branching Models at 10,000 ft

• Lots of religious debate! But many would say…– Develop most new code on the “trunk”

• Aka “main line”, aka “development branch”– Branch per feature and/or per team if absolutely

required– Branch to stabilize and release

• Feature Branch– Isolate work on new or experimental features that might

cause instability to the rest of the project. – Isolate work on interface changes that will cause

instability for the rest of the project.

Branching Models at 10,000 ft

• Team Branch– Isolate sub teams so they can work without being

subject to breaking changes from other teams. – Isolate sub teams so they can work towards unique

milestones. Team branches often have feature branches beneath them.

• Release Branch– Branch for builds that will require you to perform

ongoing service/maintenance (sub-branch for hotfixes)– Branch to pursue/stabilize multiple releases in parallel

Considerations in Branching• Consider structuring branch trees so you only need

to merge up/down the branch hierarchy – not across– Avoids need for “baseless merge” (tf /merge baseless …)– Remember that branch hierarchy likely doesn’t match

repository and file system layout in terms of the “tree”– Merging a change back “up” to main line and “down” to

another branch requires care

• If you branch too deeply (Main -> Team -> Feature -> Experimental work…) you will introduce latency

• Merges require conflict resolution – and bad merge decisions can happen, so the resulting build must be tested thoroughly

A few branching use cases…

Branching to aid nightly build stability…• Dev (aka trunk/main) branch for active check-ins

– “It compiled locally and on build server”– All developers have read/write– Continuous builds (CI) should happen here– Use for quick feedback & feature tests

• Integration branch for integration– “It passed build verification tests”– All developers have read; limited people have read/write– Only merge non-breaking changes from “Dev”– Use as basis for perf, security, integration tests (might

call “Test” branch for this reason)– Nightly builds– Can use as basis for production branches

Branching to stabilize new feature

• Dev branch may have child branches for each feature requiring active, overlapping & isolated development– Dev

• Feature 1• Feature 2• Feature 3

• Only “worth it” if your features frequently collide (broken builds and/or frequent conflicts) and…– the cost of merging conflicts in real time is higher than

the cost of merging conflicts between branches!

Branching for independent teams…

• Dev branch may have child branches for each team that requires active, overlapping & isolated development– Dev

• Team 1

• Team 2

• Team 3

• Only “worth it” if your teams frequently collide (broken builds and/or frequent conflicts) and…– the cost of merging conflicts in real time is higher than

the cost of merging conflicts between branches!– Sometimes true for geographically distributed teams

where communication overhead is high

Branching to isolate changes in external dependencies…

• “Dev” branch may have a child branch where changes to third party libraries (or internal frameworks) are absorbed & tested to ensure no breaking changes

• Merge to Dev/Main can occur once testing complete

Branching to support a release• Dev branch – everyone works here if team marching together…• Integration/Test – merge from Dev and to Production from here

(always the basis for production branch – not last release)• Production branches -

– “Release 1”• Hot fixes are likely labels on the release branch, not new branches…

– “Release 2”• If different teams are pursuing work on different releases and

they require isolation, team may work directly in release branches to pursue stabilization or service/maintenance work– Fixes can be merged back to Dev (aka trunk)– But carefully: The fix might not apply directly!

• V.next work can happen in Dev branch once the release branch has been created. Can be shelved temporarily until that happens.

Branching to support a release• Branch models like this are termed “Mainline

development” or “branch by purpose”• Note: Fixing a bug in Release 1 that also

exists in Release 2 requires careful thought– Can fix at earliest point in time (say Release 1),

merge back to trunk, then merge forward to Release 2

– But this includes risk of taking unwanted main line change into Release 2

– May wish to attempt baseless merge instead in some circumstances

Alternate branching method

• Challenges to using each release branch as basis for next release (as opposed to using mainline):– Not intuitive for developers to point their environment to

different branches (though vs.net makes this easier)– Difficult to work on multiple releases at once – even more

difficult to work on future release• Private branch will require baseless merge

– Easier to work in one branch, knowing work is promoted to appropriate branch

• Advantage is that bug fix in old release only has to be propagated forward (not up to trunk & back down, or across)

• Lots of debate on which model is better…

Branching to model promotions…

• Branch per environment – Dev, QA, Stage, Production…

• Labels mark release points – branch-by-label later if you need to for special purpose

• This method generally accompanied by fine-grained changesets with clear work-item associations

• As you merge from Dev to QA (etc.) you can hand-pick changesets– Or by work item using TeamCM

Team Foundation Promotion Modeling(Branches of varying quality)

foo.csfoo.cs

bar.csbar.cs

$/…/Dev$/…/Dev

$/…/Test$/…/Test

$/…/Prod$/…/Prod1818

1818

foo.csfoo.cs

bar.csbar.cs 1313

1313

foo.csfoo.cs

bar.csbar.cs 99

99

1010

1212

1515

1414

1616

1515

2020

1919

2222

2121

2525

2525

1717

1717

2323

2323

2626

2626

2424

2424

2727

2727

After 12: Branch to test = 13After 16: Merge latest to test = 17After 17: Branch to prod = 18After 22: Merge 21, 22 to test = 23After 23: Merge latest to prod = 24After 25: Merge latest to test=26After 26: Merge latest to prod=27

Merging Mechanics in TFS

• When you merge, you initiate from the “source” – where the changes are coming “from”– Then choose merge “target”

• Can merge by:– Latest version– Label– Date– Changeset

Merging

• Can merge by:– Selecting individual changeset(s)…

• Useful for handpicking bug fixes or changes to merge• Closed bugs (work items) will tell you which changesets are

associated fix/change• Must be a contiguous range of changes for a given merge

operation (ouch)• Keep your changesets fine-grained !

– Or use TeamCM to merge by work item

Merging• Merge includes content additions, deletions,

un-deletions, renames, moves…not just changes between two files!

• Merge operations can require conflict resolution if development is occurring in the merge target

• Note that merge requires your workspace to have “visibility” to all branches involved

• Note that merge process itself doesn’t update the target branch on your local disk – but that is where merge will take place from !

Merging• Merge operations result in a set of pending

changes (of type “merge”) which must be checked in (and will result in single changeset)

• Merge history maintained for you – system knows what the “remaining” merge candidates are from branch parent to branch child (unlike svn)

Agenda

• Branching & Merging – Mechanics within TFS– Strategies

• How does Team Build help with SCM?• Carrying through to physical environments

– TFS Deployer

SCM Mileage from Team Build• Team Build can be used in “branch aware” fashion

(i.e. a build type per branch)• Multiple build types can be created for each branch…

– In Debug, Release variants…– For continuous integration, nightly schedule, or on-demand– With different levels of post-build automated testing– Each producing a list of changesets that have been

incorporated since last build (though merge operations will appear as one changeset if checked in together)

• Depending on which CI solution you use, you can enact dependency replication as part of build– TFS Integrator will do this…– (Or use MSDN sample)

SCM Mileage from Team Build• If you have a team build type per branch…• And you have continuous integration configured

for each branch…• Then every merge between branches (Dev->Test

or FeatureX->Main) will trigger a build• The “branch build” can involve unit tests that are

assessing (in part) whether the merge operation produced good results– Valuable, since merges often error prone

CI for TFS 2005

CI for TFS 2008

Agenda

• Branching & Merging – Mechanics within TFS– Strategies

• How does Team Build help with SCM?• Carrying through to physical

environments– TFS Deployer

SCM Definition Revisited• “The process of identifying and selecting

all the elements of a software system that create the correct and cohesive set - at defined points in time.”

• Once you have a branching/merging strategy in place, and a corresponding set of builds that occur based on those branches…

• Carry it through to a set of automated deployments to corresponding physical environments!

TFS Deployer (Readify)• See http://shurl.org/tfsdeployer• Installed as an agent on servers that you must

deploy to– Dev environment, Test environment…Production?

• Listens to build quality change events– Build quality is edited within build store in vs.net– Build store notifies event service, which lets all

subscribers (including TFS Deployer) know about quality change

TFS Application/Database Server

Target Machine

TFS Build Store

TFS Event Service

TFS Event Subscription Store

Developer Machine

User running Team Explorer

2 – Update Build Quality

TFSDeployer Service

1 – Subscribe to build qualitychange events

3 – Build Quality Change Event

4 – Subscriber Information

5 – Build Quality Change Event

PowerShell Instance

TFS Version Control Store

6 – Deployment Mapping File and PowerShell Script

7 – Create Power Shell Instanceand run script

TFS Deployer (Readify)• TFS Deployer uses deployment mapping file

(checked into subfolder of team build type) to determine what (if any) action to take– Deployment file might indicate that a deployment for

test servers should be kicked off when build quality transitions from “Dev Accepted” to “Test Staging”

• Deployment Mapping file associates servers to actions based on build quality transitions for a given build– Can deploy to a farm by having multiple servers listed– Actions are powershell scripts

Team Build and “Build Quality”

• Build Quality names in Team Build are customizable – so choose to match your process!

• Mitch Denny (Readify) suggests:– Unexamined– Development Staging/Accepted/Rejected– Test Staging/Accepted/Rejected– Production Staging/Accepted/Rejected

• Transitions can be secured…

Synergies from the whole…• When you have a good branch/merge strategy…• With corresponding automated builds…(per branch)• And appropriate & integrated automated unit

tests…• With automated deployments to corresponding

physical environments…• You will find a whole host of positive team behaviors

can emerge !– Because time isn’t lost due to poor SCM (wrong software in

the wrong place, incorrectly deployed)– Because the team can treat that infrastructure as a given…– From identified bug to a traceable, farm-deployed fix in 10

minutes ?

Extending release management to database artifacts: Visual Studio for Database Pros

• Database Refactoring• Schema comparison• Data generation tools• Database unit testing tools• Database projects…

– Ability to harvest all schema objects from existing scripts or live schema

– Triggers, functions, security elements, sprocs, tables, view, constraints, indexes, keys, triggers, UDT…

– House in a single project with both “script centric” and “schema centric” views

Visual Studio for Database Pros• Key idea is to bring database development work

into the standard development lifecycle– Version control (branching/merging)– Collaboration with team– Unit tests

• Ideal: Each team member has local copy of database, where changes to sprocs, etc. made in isolation – and then unit tested

• Communication of schema/sproc changes to team and propagation through environments begins with check-in!

• “Building” a database project is an operation that is relative to a target environment– And…it generates the correct update script for you– Customization (and pre/post scripts) available for cautious

DBAs once you are targeting stage/production

Thanks -