TFS at Siemens Healthcare - Microsoft 1/Track 3/TFS... · TFS at Siemens Healthcare Best practices...

Unrestricted © Siemens AG 2014 All rights reserved. Answers for life.

TFS at Siemens Healthcare

Best practices and learning's ALM Days 2014 – Thomas Dasch & André Mainka

2014-02-24

Unrestricted © Siemens AG 2014 All rights reserved.

Page 2 Thomas Dasch – André Mainka / Healthcare, Imaging & Therapy, SYNGO

syngo.via – Get the full picture.

TFS 2012 migration

About us

Large-scale considerations

Our experience

Agenda

Operate a huge build farm

2014-02-24



About us

2014-02-24



The syngo portfolio

Seamless integration across the entire imaging workflow

1 syngo.via can be used as a stand-alone

device or together with a variety of

syngo.via-based software options which are

medical devices in their own right

2 syngo.share is a medical device of ITH

icoserve technology for healthcare GmbH,

Innsbruck. syngo.share is currently

available only in various European

countries.

2014-02-24



syngo.via – Get the full picture.

The syngo.via1 3D routine and

advanced reading solution helps to

accelerate workflows across all

modalities.

It is efficient.

It helps to accelerate radiological

workflows and save valuable time.

It is flexible.

You can customize its applications

and layouts in line with your needs.

It is intelligent.

The software guides you through

the entire workflow.

syngo.via General Engine1

1 syngo.via can be used as a stand-alone device or together with a variety of syngo.via-based software options, which are medical devices in their own right. This product is not yet

commercially available in all countries. Due to regulatory reasons it’s future availability cannot be guaranteed. Please contact your local Siemens organization for further information.

2014-02-24



Our constraints

Large scale projects have to manage various challenges

Medial devices

Regulated environment

Global development

footprint

Thousands of active users

Million lines of code

Multiple product lines

Terabytes of data

Massive number of

builds

Myriad of tests

2014-02-24



Drivers in large scale ALM

Performance Serviceability Scalability

2014-02-24



Our TFS history

Stepwise completing the ALM approach

2010 • TFS 2010

Infrastructure

2011 • Project

Management

• Backlog Planing

2012 • Build & Version

Control

• Automated Test Execution

2013 • Defect

Management in TFS

• TFS 2012

2014 • TFS 2013

• Test Management in TFS

2014-02-24



TFS 2012 migration

2014-02-24



TFS upgrade

Rollout in a large scale environment is challenging

Plan

Prepare

Test

Migrate

Monitor

Tradeoff

update cadence

vs. effort

2014-02-24



TFS 2010 to TFS 2012 upgrade

Upgrade is more than just running the installer

Precondition: TFS 2012 Update 2 made build 2010

infrastructure compatible

Prepare your migration,

e.g. by reducing the amount of data to be migrated

• Outdated build results

• Obsolete shelvesets

• Unnecessary labels

• Superfluous workspaces

• Locked files

Is your old build infrastructure compatible with the

new TFS backend?

Execute a bunch of scout tests after the migration

2014-02-24



TFS 2010 to TFS 2012 upgrade

Don‘t underestimate data migration

• Upgrade process longer than expected

• Huge amount of data increases the data migration

• Warehouse is recreated from scratch

• Upgrade schedules additional jobs

• Still a few bugs found

• QueryMerges API returns wrong results

• SymbolServer deletion does not work

• JobAgent crashes under certain conditions

• Builddefinitions compatibility with Visual Studio

versions

2014-02-24



Microsoft communication strategy

Advertising surface vs. detailed change knowledge

New features are highly advertised and widely

discussed

More crucial for installed base is detailed information

about

• Stability fixes

• Performance fixes

• Scalability fixes

• Compatibility fixes

• Customization impact

http://blogs.msdn.com/b/visualstudioalm/archive/2011/09/20/

visual-studio-team-foundation-server-11-developer-preview-

what-s-new-for-team-foundation-server.aspx

http://blogs.msdn.com/b/bharry/archive/2012/03/23/keeping-

your-tfs-2010-up-to-date-as-of-march-2012.aspx

http://blogs.msdn.com/b/visualstudioalm/archive/2011/09/20/visual-studio-team-foundation-server-11-developer-preview-what-s-new-for-team-foundation-server.aspx
































http://blogs.msdn.com/b/bharry/archive/2012/03/23/keeping-your-tfs-2010-up-to-date-as-of-march-2012.aspx























2014-02-24



Large scale considerations

2014-02-24



• Build and test infrastructure on

site

• Version control proxy

Physical network topology

Thousands of users require an advanced configuration

• Load balanced application tiers

• Dedicated hardware for

reporting and analysis services

• SQL cluster with high

availability configuration

2014-02-24



Server side customizations

Various extension points can be used to adapt the feature set

• German/European industrial relations law (BetrVG)

• Make reports anonymized

• Regulatory requires traceability

• Prevent to edit user display name

• Disable version control adapter

• Data warehouse crashes if amount of data is

extremely high

• Backend automation/aggregation Service

• E.g. accumulate story points into parent work

item

2014-02-24



SQL administration

SQL server is the mission critical resource

• Ensure that your backup procedure is operational

• Working with your amount of data

• Test the disaster: Restore backup

• Ensure that SQL configuration is optimized for

performance

• Monitor log data and free space

• Monitor Temp DB free space

• Monitor table sizes & row counts

http://blogs.msdn.com/b/granth/archive/2013/10/08/what-

does-a-well-maintained-team-foundation-server-look-

like.aspx

http://blogs.msdn.com/b/granth/archive/2013/10/08/what-does-a-well-maintained-team-foundation-server-look-like.aspx




















2014-02-24



TFS operations tasks

Regular maintenance keeps the system operational

• Deleted builds are just marked as deleted and need

to be destroyed

• Clean test attachments

• Delete shelvesets which remain by failed gated

checkin builds

• Delete unused workspaces

2014-02-24



Operate a huge build farm

2014-02-24



Real-time overview of running builds

Build explorer fulfills just a subset of the demands

View currently running builds

• Monitor queue time

• How many builds are enqueued?

• How long are builds already running?

Evaluate the history of a single build definition

• Check completed builds for irregularities

• How long does the build take in average?

• How often gets this build executed?

The overview gives you a first impression of

what’s going on in your system.

2014-02-24



Keep track of different build definitions

Aggregate the latest results on a dashboard

No „out of the box“ feature for

• Monitoring staged integration gates

• Keeping track of multiple branches

Recommendations

• Aggregate the most important details

• Audio feedback to increase awareness

• Have an overview of all your builds

• Keep everything in view, focus on the essential

• Display the status of the last build on a dashboard

2014-02-24



Analyze build details

Know exactly where the time is spent

• Fast real-time detail view for a single build

• Immediately see the context if the build has failed

• Which part of the build takes most time?

• Which area requires optimization?

• Does the load balancing of my tests work?

Customized from

http://buildexplorer.codeplex.com/

First focus on bottlenecks, before starting with

fine-tuning




2014-02-24



Build optimization

Various areas impact quality and speed

Infrastructure (Server, Network, Agents, Controller)

Teambuild Test Framework /

Test Environment

Product and Tests

2014-02-24



Hardware optimizations could be cheap

High performance setup requires fine tuning

Measure, measure, measure…

Identify physical bottlenecks:

• CPU, RAM, HDD, Network

Fine tune configurations

• Power options: high performance

• Disable Windows search indexing

• Exclude \builds, \symbols, \drop, team explorer

cache from anti-virus real time scanning

• Clean up the \builds and TEMP folders on build

agents to remove old workspaces

2014-02-24



Compile time is a significant part of build cycle time

Fast Compilation is a key to optimize performance

Compilation should fully leverage all CPU cores

Does compile use an incremental approach?

Advanced optimizations

• Reduce dependency chains

• C++ compile on multiple processors

• C++ incremental compile

• C++ precompiled Header

• Optimize critical path

Learn how MsBuild works

2014-02-24



Find a balance between testcoverage and execution time

Test execution does not scale on a single machine

• Fast feedback of Integration tests is mandatory

• Scale out especially for UI based or long running

tests

• Leverage TestController infrastructure (without lab)

to distribute tests across multiple test agents (aka

remote execution)

• Invoke directly from teambuild

http://msdn.microsoft.com/en-us/library/dd728093.aspx

Reuse/integrate existing infrastructure components





2014-02-24



Fine-tune the build process

Improve and optimize – towards release pipeline

Fine-tune build definition

• Incremental get

• Label sources

• Drop build output

• Index sources

Customize build process for advanced configuration

• Dump important settings to buildlog

• Fail fast

• Remote test execution

• Drop log files only

• Reduce „noise“ in build logs

• Enforce checkin-policies by build

2014-02-24



Cleanup build artifacts

Sophisticated Retention Policies needed

• Default retention policies are not sufficient

• „I like to keep 100 builds but cleanup the details“

• Less amount of build drops needed

• Delete oldest drops before drop location

exceeds disk space

• How to cleanup my custom shares?

• How to delete build artifacts for already deleted

builds?

• Note: Deleted builds vanish in build explorer but

remain in database

2014-02-24



Keep track of the overall Build Farm

Monitor fundamental parameters and alert before you run into a downtime

• Impossible to manually keep track of many

machines

• „I like to know that my infrastructure is operational

without being called by my customers“

• Use monitoring tools like Nagios/Icinga

• Monitor build and test machines

• Keep track of handles, memory, disk space,

latency

http://www.nagios.org/ https://www.icinga.org/

.Net 4.5.1 installed

http://www.nagios.org/

http://www.nagios.org/

https://www.icinga.org/



2014-02-24



Our experience

TFS Strengths and weaknesses

syngo.via WebViewer VA11A is not commercially available in all countries. Due to regulatory reasons its future availability cannot be guaranteed. Please contact your local Siemens organization for further details. Diagnostic

reading of images with a web browser requires a medical grade monitor. For iPhone and iPad country-specific laws may apply. Please refer to these laws before using for diagnostic reading/viewing.

Apple®, the iPhone®, the iPad®, iPod Touch® are trademarks of Apple Inc., registered in the U.S. and other countries. For iPhone, iPad, iPod Touch country specific laws may apply. Please refer to these laws before using .

2014-02-24



Our experience

TFS supports performance for large scale ALM

Performance

Strengths

• Reasonable response time

• Allows fine tuning at many places

• Supports efficient development

Weaknesses

• Some fundamental features missing out of the box

2014-02-24



Our experience

TFS supports maintainability for large scale ALM

Serviceability

Strengths

• Is highly customizable

• Customization allows to workaround issues

• Requires low administration efforts

Weaknesses

• Transparency about known issues

• Monitoring solution / administrative guidance

• Tight coupling of TFS to latest .Net framework

2014-02-24



Our experience

TFS supports scalability for large scale ALM

Scalability

Strengths

• Scales well for very large teams

• Fits for global development footprint

• Allows customized reuse of existing services

Weaknesses

• Does not work out of the box for large scale

• Some hard-constraints limit extensibility

2014-02-24



syngo – See results.

TFS at Siemens Healthcare - Microsoft 1/Track 3/TFS... · TFS at Siemens Healthcare Best practices...

Documents

Transcript of TFS at Siemens Healthcare - Microsoft 1/Track 3/TFS... · TFS at Siemens Healthcare Best practices...