Benjamin Day – VSTS/TFS Workshop · 2016-11-03 · Code Coverage Visio and UML Mod eling Team...
Transcript of Benjamin Day – VSTS/TFS Workshop · 2016-11-03 · Code Coverage Visio and UML Mod eling Team...
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 1
Getting the Most Getting the Most Mileage Out Of Team Mileage Out Of Team System: A Developer’s System: A Developer’s PerspectivePerspectiveBenjamin DayBenjamin DayBenjamin Day Consulting, Inc
About the speakerAbout the speaker
Owner, Benjamin Day Consulting, Inc.– Email: [email protected]– Web: http://www.benday.comp // y– Blog: http://blog.benday.com
Trainer– Visual Studio Team System, Team Foundation Server
Microsoft MVP for C#Microsoft VSTS/TFS Customer Advisory CouncilLeader of Beantown.NET INETA User Group
AgendaAgenda
TFS & VSTS OverviewUnit Testing & Test-Driven DevelopmentTeam ProjectsSource ControlSou ce Co t oDatabase Development with VSTSTFS Build System
Section 1: TFS & VSTS OverviewSection 1: TFS & VSTS Overview
Visual Studio Team SystemVisual Studio Team System
tect
ure
Gui
danc
ete
ctur
e G
uida
nce
Dynamic Code Analyzer
Visual Studio
Team Architect
Static Code Analyzer
Code Profiler
Unit Testing
d
Load Testing
Manual Testing
Test Case Management
Application Designer
Logical Infra. Designer
Deployment Designer
Visual Studio
Team DeveloperVisual Studio
Team Test
dust
ry P
artn
ers
dust
ry P
artn
ers
5
Change Management
Work Item Tracking
Reporting
Project Site
Visual Studio
Team Foundation
Integration Services
Project Management
Proc
ess
and
Arc
hiPr
oces
s a
nd A
rchi Code Coverage
Visio and UML Modeling
Team Foundation Client (includes CAL)
Visual Studio Professional Edition
Visu
al S
tudi
o In
dVi
sual
Stu
dio
Ind
Big Build
Class Designer
Visual Studio Team ArchitectVisual Studio Team Architect
Application DesignerLogical Infrastructure DesignerDeployment DesignerClass DesignerC ass es g eNo unit testing
6
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 2
Visual Studio Team SystemVisual Studio Team System
tect
ure
Gui
danc
ete
ctur
e G
uida
nce
Dynamic Code Analyzer
Visual Studio
Team Architect
Static Code Analyzer
Code Profiler
Unit Testing
d
Load Testing
Manual Testing
Test Case Management
Application Designer
Logical Infra. Designer
Deployment Designer
Visual Studio
Team DeveloperVisual Studio
Team Test
dust
ry P
artn
ers
dust
ry P
artn
ers
7
Change Management
Work Item Tracking
Reporting
Project Site
Visual Studio
Team Foundation
Integration Services
Project Management
Proc
ess
and
Arc
hiPr
oces
s a
nd A
rchi Code Coverage
Visio and UML Modeling
Team Foundation Client (includes CAL)
Visual Studio Professional Edition
Visu
al S
tudi
o In
dVi
sual
Stu
dio
Ind
Big Build
Class Designer
Visual Studio Team DeveloperVisual Studio Team Developer
Static Code AnalyzerCode ProfilerUnit TestingCode CoverageCode CoverageClass Designer
8
Visual Studio Team SystemVisual Studio Team System
tect
ure
Gui
danc
ete
ctur
e G
uida
nce
Dynamic Code Analyzer
Visual Studio
Team Architect
Static Code Analyzer
Code Profiler
Unit Testing
d
Load Testing
Manual Testing
Test Case Management
Application Designer
Logical Infra. Designer
Deployment Designer
Visual Studio
Team DeveloperVisual Studio
Team Test
dust
ry P
artn
ers
dust
ry P
artn
ers
9
Change Management
Work Item Tracking
Reporting
Project Site
Visual Studio
Team Foundation
Integration Services
Project Management
Proc
ess
and
Arc
hiPr
oces
s a
nd A
rchi Code Coverage
Visio and UML Modeling
Team Foundation Client (includes CAL)
Visual Studio Professional Edition
Visu
al S
tudi
o In
dVi
sual
Stu
dio
Ind
Big Build
Class Designer
Visual Studio Team TesterVisual Studio Team Tester
Load TestingManual TestingTest Case ManagementWeb Testingeb est gUnit TestingCode Coverage
10
Visual Studio Team SystemVisual Studio Team System
tect
ure
Gui
danc
ete
ctur
e G
uida
nce
Dynamic Code Analyzer
Visual Studio
Team Architect
Static Code Analyzer
Code Profiler
Unit Testing
d
Load Testing
Manual Testing
Test Case Management
Application Designer
Logical Infra. Designer
Deployment Designer
Visual Studio
Team DeveloperVisual Studio
Team Test
dust
ry P
artn
ers
dust
ry P
artn
ers
11
Change Management
Work Item Tracking
Reporting
Project Site
Visual Studio
Team Foundation
Integration Services
Project Management
Proc
ess
and
Arc
hiPr
oces
s a
nd A
rchi Code Coverage
Visio and UML Modeling
Team Foundation Client (includes CAL)
Visual Studio Professional Edition
Visu
al S
tudi
o In
dVi
sual
Stu
dio
Ind
Big Build
Class Designer
Visual Studio Database ProfessionalVisual Studio Database Professional
Released December 2006 Source control for database schemasDatabase object “rename” refactoringTest data generatorest data ge e atoDatabase unit testsSchema comparisonsTable data comparison
12
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 3
Visual Studio Team SystemVisual Studio Team System
tect
ure
Gui
danc
ete
ctur
e G
uida
nce
Dynamic Code Analyzer
Visual Studio
Team Architect
Static Code Analyzer
Code Profiler
Unit Testing
d
Load Testing
Manual Testing
Test Case Management
Application Designer
Logical Infra. Designer
Deployment Designer
Visual Studio
Team DeveloperVisual Studio
Team Test
dust
ry P
artn
ers
dust
ry P
artn
ers
13
Change Management
Work Item Tracking
Reporting
Project Site
Visual Studio
Team Foundation
Integration Services
Project Management
Proc
ess
and
Arc
hiPr
oces
s a
nd A
rchi Code Coverage
Visio and UML Modeling
Team Foundation Client (includes CAL)
Visual Studio Professional Edition
Visu
al S
tudi
o In
dVi
sual
Stu
dio
Ind
Big Build
Class Designer
The Glue: Team Foundation ServerThe Glue: Team Foundation Server
Binds all the developers together– Enables the “System” in Team System
Streamlines project management and communicationProvides a unified environment for managing– Task assignments– Progress data– Bug data– Source control– Builds– Unit test data
14
Why?Why?
15
Software projects tend to have Software projects tend to have problems…a lot. problems…a lot.
Standish Group’s “Chaos Report” from 200313522 IT projects34% of projects succeed (16% in 1994)15% of project failed (31% in 1994)5% o p oject a ed (3 % 99 )51% of projects “challenged”– 54% of “challenged” more than 20% over-budget
from Business Wire, March 25, 2003
16
Success by project size for 2000Success by project size for 2000
From “Why Big Software Projects Fail” by Watts Humphrey, CrossTalk: Journal Of Defence Software Engineering, March 2005
17
Standish Group project definitionsStandish Group project definitions
Successful project– Completed within +/- 10% of cost and schedule
estimate– All intended functions
Challenged projectsChallenged projects– Late – >10% over budget– Reduced functions
Failed Projects– Never delivered anything
18
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 4
Why do software projects fail?Why do software projects fail?
Poor requirementsRequirements shift– Scope creep– People change their minds
Disconnect between management, customer, and the development team– No unified “big picture” view
Poor task estimation (unrealistic schedule)Failure to recognize and manage risk
19
TFS and Team System: Why do we TFS and Team System: Why do we need it?need it?
Do you really want to work around the clock?Do you love “death march” projects?
Microsoft wants you to be successfulMicrosoft wants you to be successfulTest-driven development is an accepted best practiceDifficult to determine project statusTeam communication overheadMicrosoft had no “enterprise” version control solution
20
TFS: Why do we need it?TFS: Why do we need it?
Improves communication Expectations of management get out of sync with the reality that developers see Geographically dispersed teams (different offices, time zones, countries) exacerbate the communication problemTFS provides a single place where all data and artifacts of a product are stored and reported on so that all stakeholders in the product can have a shared understanding
21
Team SystemTeam System
Focused on increasing the predictability of successVisual Studio – Built-in unit testing, code coverage, code profiling
(Another unit test framework?!)Static code analysis– Static code analysis
– Integrates with Team Foundation Server
Team Foundation Server – Work item management – Reporting– Source control– Build management– Project guidance documentation
Not all features will be used by everyone role centric
22
TFS Architectural OverviewTFS Architectural Overview
TFS is an n-tier applicationClient-side (Presentation)– Visual Studio .NET 2005 Team Edition– Team Explorer
Server-side (Presentation)( )– SharePoint Services 2.0– SQL Reporting Services
TFS Web Services “Business” Tier– Team Foundation Server
Database Tier– SQL Server 2005– 11 Databases
23
Team ExplorerTeam Explorer
Plug-in for Visual Studio Team SystemYour interface to TFSNot part of the VSTS installationInstall it from the TFS disksta t o t e S d s
24
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 5
Team ProjectTeam Project
Similar to a “solution” in Visual StudioHas a development process (aka methodology)– Agile, CMMI, or Scrum
Container for all items related to the project – Work items (TFS)– Documents (Sharepoint)– Reports (SQL Reporting Services, Excel)– Builds (TFS)– Source Code (TFS)
25
Questions?Questions?
Section 2: Unit TestingSection 2: Unit Testing
What is unit testing & TDD? What are all different test types? How do you design for testability? What’s a “mock” object? at s a oc objectHow do you test user interface functionality? What’s code coverage? Why would you want to use code profiling on your unit tests? What’s the best way to unit test your stored procedures?
What is a Unit Test?What is a Unit Test?
Wikipedia says, “a unit test is a procedure used to validate that a particular module of source code is working properly”Method that exercises another piece of code
d h k lt d bj t t t iand checks results and object state using assertions
28
What is Test Driven Development?What is Test Driven Development?
Way of developing code so that you always have proof that something is working– Code that validates other code– Small chunks of “is it working?”
S ll h k U it T tSmall chunks = Unit TestsKent Beck (“Test-Driven Development”, Addison-Wesley) says “Never write a single line of code unless you have a failing automated test.”
29
How to begin?How to begin?
Get an idea for what you want to developBrainstorm success and failure scenarios– These become your tests
30
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 6
The ProcessThe Process
From William Wake’s “Extreme Programming Explored” (Addison-Wesley)
1. Write the test code2. Compile the test code Fails because there’s no
implementationimplementation3. Implement just enough to compile4. Run the test Fails5. Implement enough code to make the test pass6. Run the test Pass7. Refactor for clarity and to eliminate duplication8. Repeat
31
Basic structure of a Visual Studio Basic structure of a Visual Studio unit testunit test
Microsoft.VisualStudio.QualityTools.UnitTestFramework
Test fixture (aka Test Class)– Marked with the [TestClass] attribute
Collection of unit test methods– Collection of unit test methods– Needs a parameter-less constructor
Unit test method – Marked with the [TestMethod] attribute– Must be public– Must return void– No method parameters
32
DemoDemo
Using the AdventureWorks databaseWrite a test to create a new person
33
Why Use TDD?Why Use TDD?
High-quality code with fewer bugs– The bugs you find are easier to diagnose
Using the “test first” method means you think out how your code will work ~up-front designdesignLess time spent in the debugger– Do you remember what it was like when you first
started doing OO code?
And because you have tests that say when something works…– Easy to maintain & change Refactoring– Code is exercised and the unit tests document how
the developer intended it to be used Self documenting 34
Why you shouldn’t test: ExcusesWhy you shouldn’t test: Excuses
“It takes too much time”– Wrong: it does take time to create/run unit tests, but it
will reduce the overall time spent fixing defects in the code
– Code that’s well-tested can be modified / extended very ileasily
– Code that’s not tested can’t be reliably built-onto– Unit Tests let you pay-as-you-go rather than try to
test/debug/fix more complex interrelated code much later
– “Dude, pay now or pay later. It’s up to you.”
35
Why you shouldn’t test: ExcusesWhy you shouldn’t test: Excuses
Here’s what takes too much time:– Time spent debugging (your own or other’s code)?– Time spent fixing defects in code that previously was
assumed to be working– Time spent isolating a reported bug
Time needed to write Unit Tests will only consume a small amount of the time freed when you reduce time spent on these
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 7
Why you shouldn’t test: ExcusesWhy you shouldn’t test: Excuses
“It takes too long to run Unit Tests”– Wrong: most unit tests run very quickly– You can run dozens or hundreds of tests in seconds– You can always separate out long-running tests
“I d ’t k h th d i d t“I don’t know how the code is supposed to behave, so I can’t test it”– WHAT?!– If you don’t know how it’s supposed to behave you
shouldn’t be writing it!
“I’m not allowed to run tests on a live system”– Wrong: Unit Tests aren’t for testing a live system.– Unit tests are for testing by developers using your
37
Why you shouldn’t test: ExcusesWhy you shouldn’t test: Excuses
“It’s not my job: I’m supposed to write code”– Wrong: you’re supposed to write reliable code– Until you’ve objectively verified its reliability – and
done so in a way that can be repeated easily – you’ve not done your joby j
“I’m being paid to write code, not write tests”– Wrong: you’re being paid to write dependable,
working code, and not to spend the day debugging it.– Unit Tests are means to guarantee that your code is
working.
“I can’t afford it”– Wrong: you can’t afford not to test– Plus, you can’t perform serious development of .NET
and Windows without MSDN Premium (includes VSTS38
Do I have to?Do I have to?
Yes, and you’ll be glad you did. You’ll thank me when you’re older– As your system gets larger, you’ll be glad you wrote
your tests
A t t d M h i th li kiAutomated Much easier than clicking through 80 different screens to exercise your code or re-create a bug Automated regression testing– Since unit tests (ideally) run quickly, as you develop
you can test how your changes affects other people’s code – not just yours
39
More unit test structureMore unit test structure
Assembly setup/teardown– Executed once per test assembly– [AssemblyInitialize]– [AssemblyCleanup]
Fixture setup/teardownFixture setup/teardown– Executed once per test fixture– [ClassInitialize]– [ClassCleanup]
Test setup/teardown– Executed for each test– [TestInitialize]– [TestCleanup]
40
Other unit test attributesOther unit test attributes
[Ignore]– Class or method attribute– Causes unit test framework to skip the test or fixture
[ExpectedException]– Method attribute– Used to test error handling– Test fails if an exception of the expected type has not
been thrown
[DeploymentItem]– Method attribute– Specifies a file that should be copied to the test run bin
directory
[Timeout]– Method attribute– Maximum time in milliseconds the test should run
41
Assert MethodsAssert Methods
Use assertions to check return values, object state to confirm your coding assumptionsAreEqual / AreNotEqual– Compares by value– AreEqual() for floats and doubles have an optional
“d lt ”“delta”Passes if (Abs(expected – actual) < delta)
AreSame / AreNotSame– Compares by reference
IsTrue / IsFalseIsNull / IsNotNullIsInstanceOfType / IsNotInstanceOfTypeInconclusiveFail 42
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 8
StringAssert MethodsStringAssert Methods
Contains(string, substring)– String contains a substring– Case-sensitive
StartsWith() / EndsWith()Matches() / DoesNotMatch()– Uses regular expressions
43
CollectionAssert MethodsCollectionAssert Methods
AllItemsAreInstancesOfType()AllItemsAreNotNull()AllItemsAreUnique()– No duplicate values (value types) or instances (ref types)
AreEqual() / AreNotEqual()– Same items, # of occurance, same order
AreEquivalent() / AreNotEquivalent()– Same items, same # of occurrences– Not necessarily the same order
Contains() / DoesNotContain()IsSubsetOf() / IsNotSubsetOf()– All values in collection x exist in collection y 44
Assert methods and error messagesAssert methods and error messages
All assertion methods take an error message– Displayed if the assert fails
All methods also have an override that takes a parameterized string
()– Same as String.Format()– Assert.AreEqual(
“asdf”, “xyz”, “Values were not equal at {0}.”,
DateTime.Now);
Best practice: provide a descriptive error message on all asserts
45
Things you can do with your unit Things you can do with your unit teststests
Code CoverageCode Profiling
46
Code CoverageCode Coverage
Keeps track of which lines of code are run– More importantly, NOT run
Shows you which code your tests aren’t exercising
k l d bl & b h h– Like plaque dye tablets & brushing your teeth– Shows you all the spots you missed
Helps find unused code– Especially after refactoring– “Dead” code
47
Code Coverage DemoCode Coverage Demo
48
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 9
Code ProfilingCode Profiling
Similar to code coverageShows you the relative effort expended in different parts of your codeProfiling Modes– Sampling
Overview mode use to find performance problemsProfiler gathers broad info at intervals
– InstrumentationSpecific mode use to dig into performance problems
Microsoft recommends profiling “Release” builds
49
Why profile a unit test? Why profile a unit test?
Why profile a unit test? Why not just profile your app?Unit tests give you a predictable, repeatable usage of your app– Run your test– Look at the profiling output– Refactor for performance– Rinse, repeat
Since you have your tests, you’ll know if your performance refactoring just broke something else
50
Code Profiling DemoCode Profiling Demo
51
Best PracticesBest Practices
Write tests before writing code implementationMake tests autonomous– Avoid creating dependencies between tests– Tests should not have to run in a particular order
One test fixture ([TestClass]) per class([ ]) p– Simplifies test organization– Easier to choose where to locate each test
Avoid creating machine-dependent tests– E.g., tests dependent on a particular directory path
Use mock objects to test interfaces– Objects in test project that implement an interface to
test required functionality
Verify (and re-verify) that all tests run successfully before creating a new test 52
Best Practice: Design For Best Practice: Design For TestabilityTestability
Get as much code as possible out of the user interface– N-Tier Design – Great for re-use
InterfacesInterfaces– As much as possible, code against interfaces rather
than concrete classes– Makes it easier to create mock objects
Mocks– Lightweight, dummy objects that do exactly what you
want without complex setup
Is possible, make a method “protected” rather than private
53
Best Practice: Team Dev, TDD, & Best Practice: Team Dev, TDD, & Source ControlSource Control
Avoid “broken” builds When you’re doing TDD with a team, before you check in your changes– Do a “get latest”
G h ’ h d i d kiGet what’s changed since you started working– Rebuild
Makes sure that what you’ve changed / written still compiles when combined with other people’s code
– Retest by running the unit testsMakes sure that your tests and everyone else’s tests still pass when combined with other people’s code
– Check in your code
54
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 10
Best Practice: Bug fixingBest Practice: Bug fixing
When bugs are assigned– Probably be bugs visible from the user interface
Before you fix the bug… – Write a test to reproduce it
h h ld f l– The test should fail
Fix the bug using the unit test– Code until the test passes– Run all the other tests to check that the fix didn’t
break other parts of the system
55
What makes a good unit test?What makes a good unit test?
Exercise your code for success and failure– Code coverage
Try to write your tests so that they can run individually
l d– No particular order– No dependencies between tests
Database-related tests– Create your test data as part of the test run
You need data to exercise the “weird” cases and the only way to be sure you have “weird” data is to put it there yourself
56
Other Test Functionality in VSTSOther Test Functionality in VSTS
Generated Unit TestsMore types of unit tests– “Regular”– Data-driven
d d– Ordered– Web test (VSTS Tester Edition)
57
Generating Tests in VSTSGenerating Tests in VSTS
VSTS gives you tools to generate unit tests for existing codeBig Negative: Violates “test first” developmentPositive: Helps create tests for legacy codePositive: Helps create tests for non-public codeNegative: Not great looking code– Creates VSCodeGenAccessors class for non-public
members
Positive: Better than nothing
58
DataData--Driven unit testsDriven unit tests
Unit test that runs off of a data source Test executed once for each record the source– 80 records 80 test runs
2 illi d 2 illi– 2 million records 2 million test runs– Different data for each run
Helps separate the test logic from the data needed to run the testMore runs + more data more confidence in the test
59
DataData--driven test: Structuredriven test: Structure
Structure– [DataSource()] attribute– Uses TestContext object– Accesses the row data via TestContext.DataRow
60
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 11
DataData--driven test: Data Sourcesdriven test: Data Sources
Any ADO.NET data source– SQL Server– Excel– Access
M t b t bl N i ll dMust be a table No queries allowed– Impractical to use a 2 million record table as the
source
Specified on the test using the [DataSource] attribute
61
DataSource attributeDataSource attribute
[DataSource(sourceName)] – Named data source configured in app.config
[DataSource(connStr, tableName)– OLEDB connection string– Name of the source table– Name of the source table
[DataSource(providerName, connStr, tableName, method)– Provider name (e.g. “System.Data.SqlClient”)– Provider connection string– Source table– method DataAccessMethod enum
SequentialRandom
62
DataSource Configuration in DataSource Configuration in App.configApp.config
Add “microsoft.visualstudio.testtools” configuration section handler<connectionString> for each connectionAdd <dataSources> to <microsoft.visualstudio.testtools> section for each named data source
63
TestContext classTestContext class
Abstract class set to the TestContext property on tests by the testing frameworkHolder for information about the test– DataSource, DataRow– TestName– Test directory info– BeginTimer(timerName) / EndTimer(timerName)
Specify named timers timer stats get stored with the test run
– AddResultFile(filename)Include files in the test run results
64
Ordered TestsOrdered Tests
Not really a unit testTest list composed of other unit tests – Appears in the test view as one test
Arranged to run in a particular orderNo initialize or teardown methods
65
Dynamic Mock Objects with NMockDynamic Mock Objects with NMock
Mocks– Lightweight, dummy objects that do exactly what you
want without complex setup– Extend other objects or implement an interface
NMockNMock– Open-source utility – http://www.nmock.org– Creates classes at runtime that implement an
interface– Allows you to specify return values and expected
behavior for methods on the interface
66
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 12
NMock.DynamicMockNMock.DynamicMock
DynamicMock mock = new DynamicMock(System.Type);
Expect(methodName)– Expects that this method is called once
E A dR ( h dN V l )ExpectAndReturn(methodName, returnValue)– Expects this property/method is called returns the
value
ExpectAndThrow(methodName, exception)– Expects a call on this property/method throws the
exception
Verify()– Called at the end of the test– Checks that all the “expect” methods were called
67
Designing for UI TestabilityDesigning for UI Testability
Unit Testing UI’s OverviewUnit Testing UI’s Overview
Review: Why do you care about unit testing?Why are UI’s so difficult to test?How do you organize your code for UI testing?
Disclaimer: I’m a web guy.
Why Use TDD?Why Use TDD?High-quality code with fewer bugs– The bugs you find are easier to diagnose
Using the “test first” method means you think out how your code will work ~up-front designLess time spent in the debuggerLess time spent in the debugger– Do you remember what it was like when you first
started doing OO code?
And because you have tests that say when something works…– Easy to maintain & change Refactoring– Code is exercised and the unit tests document how
the developer intended it to be used Self documenting
70
Maximize Your QA StaffMaximize Your QA StaffYou shouldn’t need QA to tell you your code doesn’t workUnit testing to minimizes the pointless bugs– “nothing happened” – “I got an error message” + stack trace– NullReferenceExceptionNullReferenceException
QA should be checking for– Does meet requirements– Usability problems– Visual things (colors, fonts, etc)
When you get a bug assigned to you it should add business value
User Interfaces: The Redheaded User Interfaces: The Redheaded Stepchild of the Unit Testing WorldStepchild of the Unit Testing World
Not easy to automate the UI testingBasically, automating button clicksUI’s almost have to be tested by a human– Computers don’t understand the “visual stuff”– Colors, fonts, etc are hard to unit test for– “This doesn’t look right” errors
The rest is:– Exercising the application– Checking that fields have the right data– Checking field visibility
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 13
UI Testing Tools Are Out There…UI Testing Tools Are Out There…
For Windows forms they’re expensiveFor web applications– Visual Studio Team System Web Tests– NUnitAsp
…but they aren’t great
VSTS Web TestsVSTS Web Tests
Record paths through your applicationFills in form valuesClick buttonsValidatesValidates
Difficult to do test-driven development
NUnitAspNUnitAspOpen sourcehttp://nunitasp.sourceforge.net/Use helper objects to create code based representation of your ASPX pages, UI controlsWorks ok until you start doing custom UserControlsDownside: you basically re-create your “code-behind” with NUnitAsp objectsBig downside: Hasn’t been updated in a long while
My $0.02My $0.02
Solve the problem by not solving the problemFind a way to minimize what you can’t automate
The Solution.The Solution.
Keep as much logic as possible out of the UI– Shouldn’t be more than a handful of assignments– Nothing smart– Real work is handled by the business tier
T t th b i tiTest the business tier“Transaction Script” Pattern“Domain Model” Pattern“Service Layer” Pattern“Model View Controller” Pattern
Poll: Show of handsPoll: Show of hands
How many of you think you’ve gotten as much logic as possible out of your UI’s already?
Another $0.02: I’d bet you could get even more logic out of your UI’s.
Ok. Maybe. But how?
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 14
TieringTiering Up: Keep Logic Out Of The UIsUp: Keep Logic Out Of The UIsBusiness Object Tier (Domain Model pattern)Business Façade Tier (Service Layer pattern)– Create new Business Object methods (Factory
methods)– Wrap CRUD operations, abstract away data access
logicg– Duplicate name checks
Create an interface to represent each page in your applicationCreate Editor Facades as an adapter between the UI interfaces and the business objects
Interface interfacesInterface interfacesInterface represents the fields manipulated through the UIASPX Page or Windows Form Implements the interface– Interface’s properties wrap UI widgets– ICustomerDetail CustomerNameICustomerDetail.CustomerName m_textboxCustomerName.Text
Use a “stub” class in your unit test to represent the UIWrite unit tests to test the functionality of the editor façadeAvoid business objects favor scalars
Editor facadeEditor facade
Similar to business object facades that wrap CRUD operationsWrap larger operations that are relevant to each UI page/screen interface
i i li l k( C il)– InitializeBlank(ICustomerDetail)– View(ICustomerDetail)– Save(ICustomerDetail)
Since each page implements the interface, pass the page reference to the Editor facade
Why is this more testable?Why is this more testable?Each page/screen only has to get/set the value from interface property into the right display controlUI does not know anything about business objectsDoesn’t know about any details of loading orDoesn t know about any details of loading or savingDoesn’t have to know about validation
All this logic goes into the editor façade and testable via unit test
Avoid Referencing Business Objects Avoid Referencing Business Objects in the UI “interfaces”in the UI “interfaces”
ASP.NET– Easier to write stateless pages (everything is in
ViewState)– No need to try to re-create complex objects in code
behind in order to save
Code DemoCode Demo
Refactor to UI InterfacesPopulate drop down listsGetting/setting selected value(s) from– Drop down list– Checkbox list
Search for a value in the dbCreate new / Update existing
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 15
Questions?Questions? Section 3: Team ProjectsSection 3: Team Projects
Create a Team ProjectProcess TemplatesWork ItemsEditing with ExcelEditing with ExcelCustomizing Work Items
Team ProjectTeam Project
Similar to a “solution” in Visual StudioHas a development process (aka methodology)Container for all items related to the project – Work items– Documents– Reports– Builds– Source Code
87
Supported TFS Development Supported TFS Development ProcessesProcesses
MSF for Agile Software DevelopmentMSF for CMMI Process ImprovementScrum for Team System– http://www.scrumforteamsystem.com– TFS process template developed by Conchango
88
What is a work item?What is a work item?
Unit of work tracked by TFS Agile: Scenario, Bug, Task, Quality Of Service Requirement, RiskCMMI: Bug, Change Request, Issue, Requirement, Review, Risk, TaskHas “state” (aka status) and state transitions– e.g. Active Closed
Assignable to one personLinkable to other work itemsHas history (auditing)
89
Work Item QueriesWork Item Queries
SELECT statement against the work item databaseCan be visible to either the whole team or just individuals
90
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 16
DemoDemo
Create some work items in Team ExplorerRun a work item queryExport results in ExcelPublish changes from ExcelPublish changes from ExcelEdit a work item query
Editing Work Item TemplatesEditing Work Item Templates
Work items are defined via XMLExport / import using witexport.exe / witimport.exe
92
Demo: Edit a Work Item TemplateDemo: Edit a Work Item Template
Add the WITs to source controlAdd a field
Questions?Questions?
Section 4: Source ControlSection 4: Source Control
What is it and why use it?Set up the repositoryWhy do you care about broken builds?What is “continuous integration”?at s co t uous teg at oHow do you keep people from breaking the build?Branching & MergingShelving
Why use source control?Why use source control?
Minimize / eliminate lost workReproducible builds & product state
96
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 17
TFS Source ControlTFS Source Control
Real, enterprise-quality source controlUses SQL Server 2005 as the repositoryTransactional, atomic
97
TFS Source Control: FeaturesTFS Source Control: Features
Workspaces– Area on local disk where you edit files
Check in / check out– Check out marks the beginning of your edits– Check in commits your changes to the repository– TFS allows shared check outTFS allows shared check out
Changesets– Group of changes that happen when you check in
Shelving– Similar to check in – Changes get stored on the server – Not visible as part of the main project source tree
Branching– Used to manage multiple versions of a product
98
Modifications & Check in’sModifications & Check in’s
All these operations are batched– Add, Delete– Moves, Renames– Modifications
Branches / Merges– Branches / Merges
Batch is “sent” at check inCheck in is atomic Changeset
99
TFS Source Control: Support beyond TFS Source Control: Support beyond VS2005VS2005
Team Foundation Server MSSCCI Provider – Available for download from microsoft.com– Now on version 1.2
Source control onlySupports:Supports:– Visual Studio .NET 2003 – Visual C++ 6 SP6 – Visual Visual Basic 6 SP6 – Visual FoxPro 9 SP1 – Microsoft Access 2003 SP2 – SQL Server Management Studio – Sparx Systems Enterprise Architect 6.1 – Sybase PowerBuilder 10.5
T d f SQL S 2 0100
Demo: Add a project to source Demo: Add a project to source controlcontrol
101
Demo: Configure check in rulesDemo: Configure check in rules
102
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 18
TFS does more than just checkTFS does more than just check--in in and checkand check--outout
Branching and merging– Facilitates simultaneous development of multiple
versions of an app
Best Practice: Do not add sources directly to the root of your Team Project source controlthe root of your Team Project source control tree– $/My Team Project/Trunk– $/My Team Project/Branches
Branching & MergingBranching & Merging
You can specify 3rd-party merge tools– Tools Options Source Control Visual Studio
Team Foundation File Extensions… Add… Configure Tool
104
TF.exeTF.exe
Command-line interface to TFS source control30+ sub commands– Kind of like “net” command in Windows
Why would you want to use the command line version?1. It’s cool2. Some things aren’t available through the UI3. Good for automated operations (builds, etc)
105
The CommandsThe CommandsAddBranch / BranchesChangesetCheckin / CheckoutConfigureDelete / Undelete
MovePermissionPropertiesRenameResolveShelve / UnshelveDelete / Undelete
DirGetHistoryLabel / Labels / UnlabelLockMerge / Merges
Shelve / UnshelveStatusUndoViewWorkfoldWorkspace / Workspaces
106
Things not easily done through the GUI Things not easily done through the GUI
Find files in TFS by name/wildcard– tf dir
Find all the files that are checked out or have pending changes for the whole project
f– tf status
Get particular version of a file by wildcard– tf get
Do a preview of “Get Latest”Change the user associated with a workspace– tf workspace
107
TF.exe WorkspacesTF.exe Workspaces
/format:brief, /format:detailed/updateUserName – change the network username associated with workspaces/updateComputerName – change the name of the client computer/server – name of the TFS machine/owner – who’s workspace/computer – which computer does the workspace exist on?
108
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 19
TF.exe WorkspaceTF.exe Workspace
/new – create new workspace/template -- copy the setup of this workspace from another workspace/delete – delete the workspace/computer – computer that the workspace should/computer computer that the workspace should be created on/comment – workspace comment description– Comments can also come from a file (@commentFile)
/server – TFS machine that will govern the workspace (you need this if you access >1 team server from this computer)/newname – renames the workspace/noprompt – hide the dialog box (you’ll want this on everything)
109
TF.exe GetTF.exe Get
Gets file(s) from server to workspace/version/all – forces get all files /overwrite – wipes out read-only files that aren’t checked outchecked out/force – equivalent of /all + /overwrite/preview – show what would happen but don’t do it/recursive/noprompt – no visual dialog boxes
Example: get everything for this workspace for changeset #29– tf get * /all /version:c29 /recursive /force
110
TF.exe Switches: /versionTF.exe Switches: /version
Available on “branch”, “dir” and “get”/version– /version:1234 – by version #– /version:D10/11/2001 – by date (mm/dd/yyyy)
/ b h– /version:C1234 – by changeset #– /version:Llabeltext – by label– /version:T – latest version– /version:Wworkspacename
111
Customizing Version ControlCustomizing Version Control
Create a custom check-in policyExtend PolicyBase– Microsoft.TeamFoundation.VersionControl.Client.dll
Mark class as [Serializable]PolicyBase.Evaluate() lets you examine– What’s being checked in– Associated work items– Check-in comments– Other check-in policies
Installing the CheckInstalling the Check--in Policyin Policy
Compile Copy to the serverGo to HKLM\SOFTWARE\Microsoft\VisualStudio\8.0\TeamFoundation\SourceControl\Checkin PoliciesAdd new “string value”– Value name must be the same as the DLL name
(minus “.dll”)– Data is the full path to the DLL
Policy Gotcha!Policy Gotcha!
Policies are evaluated on the clientPolicy DLL must be installed on every developer’s computerServer-side policy configs are stored using binary serialization– Everyone must have the same version of the policy
DLL
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 20
Demo: Custom PolicyDemo: Custom Policy Demo: Static Code AnalysisDemo: Static Code Analysis
Extra Demo: Use Custom SCA Rule Extra Demo: Use Custom SCA Rule to Enforce Variable Namingto Enforce Variable Naming Questions?Questions?
Section 5: Visual Studio Team Section 5: Visual Studio Team Edition for DB ProfessionalsEdition for DB Professionals
PMO
CIO
Architect Tester
Visual Studio Team Visual Studio Team SystemSystem
Business
Analyst
Operations
Application
Support
ProjectManagerDesigner
Developer
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 21
Visual Studio Team Visual Studio Team SystemSystem
PMO
CIO
Architect Tester
Team Edition for Team Edition for Database ProfessionalsDatabase ProfessionalsE d t d t b tE d t d t b t
Business
Analyst
Operations
Application
Support
ProjectManagerDesigner
Developer
DB Pro
•• Expand to database teamsExpand to database teams•• Manage Database ChangeManage Database Change•• Extend Team productivity and Extend Team productivity and
collaborationcollaboration•• Integrated qualityIntegrated quality
What MSFT Heard from CustomersWhat MSFT Heard from Customers
aka. “why did they build it…”Managing database change is hard….
A rollback means a LONG nightDevelopment teams can end up working with out-of-date versions
Loss of revenue because the release wasn’t synchronizedLoss of revenue because the release wasn t synchronizedFinding errors at the end of the development cycle
More costly than finding them earlyIncreased support cost when you break an application from a database update
Disconnect between development and database teamsNeed to be more integrated
Conceptual OverviewConceptual OverviewDifficult to manage change to the schemaProduction database is one version of the truth for data and schemaDatabase administrator (DBA) doesn’t have access to changes until he/she has deploy or reject choiceChanges often made to production database and not rolled back into test
Production Production DatabaseDatabase
Production Production DatabaseDatabase
Management Management StudioStudio
TuningTuningMonitoringMonitoring
“One Version of the “One Version of the Truth” for Truth” for Data and Data and SchemaSchema
SchemaSchemaSchema ChangesSchema Changes
Conceptual OverviewConceptual OverviewSchema change now managed in Visual Studio Team System and Team Foundation ServerProduction Database is now “One version of the truth” only for DataDBA doesn’t have access to changes until he/she has deploy or reject choice“One Version of the truth for Schema” is Under Source Control
Production Production DatabaseDatabase
Production Production DatabaseDatabase
Management Management StudioStudio
TuningTuningMonitoringMonitoring
“One Version of the “One Version of the Truth” for Truth” for DataData
“One Version of the Truth” “One Version of the Truth” for for Schema Schema
•• Offline Offline •• Under Source ControlUnder Source Control
SchemaSchema
Schema ChangesSchema Changes
Changes can be rolled out Changes can be rolled out in a scheduled, managed in a scheduled, managed waywayScripts allow Scripts allow administrators to mange administrators to mange change updateschange updates
Database ProjectsDatabase Projects
Creating a BaselineCreating a Baseline
Creating a ProjectCreating a Project
• Core concept: offline database development• Simply a series of files collected together into a single
logical collection• The files represent the truth of your schema• Can be included in complete solutionCan be included in complete solution• Connects to SCCI providers for versioning such as Team
Foundation Server
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 22
Offline DevelopmentOffline Development
• Import database schema to populate project from existing database
• Changes to schema traditionally have immediate affect
• With offline project nothing changes until youWith offline project nothing changes until you deploy the change
Code DemoCode Demo
Create some work items in TFSImport the northwind schemaCheck it in to TFSAdd a new lookup tableAdd a new lookup tableCreate a post-deployment script
Establish the project environmentEstablish the project environment
SS
SCM
129
DatabaseDatabaseProjectProject Import schemaImport schema
DBADBA
Check in to
Check in to
Source Control
Source Control
DBDevDBDev
ProductionDatabase
StagingDatabase
Isolated Iterative DevelopmentIsolated Iterative DevelopmentSandbox
Sandbox
Sandbox
Sandbox
SCM
•• SyncSync•• CheckCheck--outout•• Edit/RefactorEdit/Refactor•• TestTest•• CheckCheck--inin
•• Work is being Work is being drivendrivenand tracked viaand tracked viawork itemswork items
130
DBADBADBDevDBDev
ProductionDatabase
StagingDatabase
Build Cycle Build Cycle
SCM
Daily BuildDaily Build Test Test
Get LatestGet Latest
TestDatabase
DailyBuild
Output
Can also beused in a “Continuous”build environment
131
DBADBADBDevDBDev
ProductionDatabase
StagingDatabase
Deploy the project environmentDeploy the project environment
SySy
SCM
132
DatabaseDatabaseProjectProject
DBADBA
ync from Label
ync from Label
DBDevDBDev
ProductionDatabase
StagingDatabase
SQLDeploy ScriptBuildBuild
DeployDeploy
Refine deploy scriptRefine deploy scriptVerifyVerify
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 23
Benefits of ApproachBenefits of Approach
Managed, project oriented evolution of database Managed, project oriented evolution of database schemaschemaApplication and database schema can now be Application and database schema can now be managed togethermanaged togetherWork in “isolation” deploying only when changesWork in “isolation” deploying only when changesWork in isolation , deploying only when changes Work in isolation , deploying only when changes verified through empirical meansverified through empirical meansLeverage VSTS work item tracking and process Leverage VSTS work item tracking and process guidance increases team collaboration and unityguidance increases team collaboration and unity
133
Ensure StabilityEnsure Stability
Testing Your SystemTesting Your System
A Rollback Means a LONG NightA Rollback Means a LONG Night
• Unit testing helps ensure that changes do not break existing code
• Unit test designer is SQL focused• Work in the language of your choice: Transact-SQL (T-SQL),
Microsoft® Visual Basic® Microsoft® Visual C#®Microsoft® Visual Basic®, Microsoft® Visual C#®
• Builds on existing Team Test Unit Test functionality
What We Can TestWhat We Can Test• Stored Procedures• Functions• Triggers• Arbitrary SQL• Support at Release to Market (RTM) to automatically
deploy changes to test system and generate data• Deterministic data generation ensures stable test
state• Can test with your application tier because of
common framework
Test DataTest Data
• To create a solid foundation for testing we support data generation
• Deterministic – always generate the same layout• Matched to your schema and very customizable• Extensible mechanism, build your own generatorsExtensible mechanism, build your own generators• Feature: DataGenerator
Generate Test Data Generate Test Data with Regular Expressionswith Regular Expressions
Names: [A-Z][aeiou][a-z]*Email Addresses:[a-z0-9]*@[a-z0-9]*\.(com|org|net|co\.uk)Phone Numbers:\([0-9]{3}\)-[0-9]{3}-[0-9]{4}
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 24
Code DemoCode Demo
Create some work items in TFSCreate some test dataWrite a stored procedureUnit test the stored procedureUnit test the stored procedure
Database ProjectsDatabase Projects
Managing ChangeManaging Change
Managed ChangeManaged Change
• Changes are local to project• Project can be compared with database• All elements can be managed under version control
• Any SCCI compliant version system
• Template driven • Version specific Microsoft® SQL Server™ 2000 or SQL Server™
2005
Working with the ProjectWorking with the Project
• Add new elements • Modify existing elements• Delete items• Deploy new or incremental update
RefactoringRefactoring
• Brings power of refactoring to SQL• Cascading change
• Update all dependent objects in database project • Schema objects, Data generation, Unit Tests, SQL Scripts
• Make an atomic change, see preview • Rename
• Meet corporate standards• Better express semantic intent – clarity
Code DemoCode Demo
Create some work items in TFSRefactor
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 25
Code Demo: Code Demo: Do Things The Wrong WayDo Things The Wrong Way
Well…if you mustModify the production schemaImport production changes into offline copy
Questions?Questions?
Section 6: Team BuildSection 6: Team Build Build/DeployBuild/Deploy• Standard Visual Studio build task• Configurations
• New versus update builds• Project properties for build• Schema compare used for build
• Pre/Post Deployment scriptsPre/Post Deployment scripts• Build results in SQL script file• Deploy
• Deploy via SQL query tool• Deploy via MSBuild task• SQLCMD command support
Why do I care?Why do I care?
Builds should be repeatableUnrepeatable low qualityAutomated easy to runEasy to run do it more oftena y o u do o oMore often continually integratingContinually integrating you know when it’s broken
149
Demo: Create a team buildDemo: Create a team build
Create a test listCreate a build with build verification tests
150
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 26
Structure of your Team BuildStructure of your Team Build
Team Build directory– Off of root of your source tree
TFSBuild.proj– The build script
TFSBuild.rsp– “Command line” arguments fed to the team build– Example: /verbosity:diag
WorkspaceMapping.xml– Used by “InitializeWorkspace” to set up source control
Demo: Run the buildDemo: Run the build
Run the buildVote of build quality
152
Demo: How to edit an existing buildDemo: How to edit an existing build
Set up workspace to get the build definitionView, check out, edit, check in
153
Run tests without test listsRun tests without test lists
.vsmdi’s are a painBuck Hodges’ Test Tools Task– http://blogs.msdn.com/buckh– http://blogs.msdn.com/buckh/attachment/951614.ashx
Runs all unit tests in an assembly
Deploying the power toyDeploying the power toy
Extract the zipCopy Microsoft.TeamFoundation.PowerToys.Tasks.QualityTools.dll to C:\Program Files\Microsoft Visual Studio 8C:\Program Files\Microsoft Visual Studio 8 \Common7\IDE\PrivateAssemblies\Copy Microsoft.TeamFoundation.Build.targets toC:\Program Files\MsBuild\Microsoft\VisualStudio\v8.0\TeamBuild
Modifying your build script to use Modifying your build script to use the power toythe power toy
Open TeamBuild.projFind
<MetaDataFile Include="$(SolutionRoot)\MyTestLists.vsmdi">
<TestList>All Tests</TestList>
</MetaDataFile>
Replace with <TestContainerInOutput Include=“MyTestAssembly.dll" />
To run with code coverage, add <RunConfigFile>$(SolutionRoot)\MyTestRunConfig.testrunconfig</RunConfigFile>
to the first <PropertyGroup> in the build script
Benjamin Day – VSTS/TFS Workshop
© 2007 Benjamin Day Consulting, Inc.. All rights reserved. 27
MSBuildMSBuild TargetsTargetsThink “events”
1. BeforeEndToEndIteration2. AfterEndToEndIteration3. BuildNumberOverrideTarget4. BeforeClean5. AfterClean6 BeforeGet
10.BeforeCompile11.AfterCompile12.BeforeTest13.AfterTest14.BeforeDropBuild15 AfterDropBuild6. BeforeGet
7. AfterGet8. BeforeLabel9. AfterLabel
15.AfterDropBuild16.BeforeOnBuildBreak17.AfterOnBuildBreak
Controlling Builds From The Controlling Builds From The Command LineCommand Line
TFSBuild.exe– start servername teamproject buildname– stop servername teamproject buildnumber– delete servername teamproject buildnumber
Desktop BuildsDesktop Builds
Run your build script outside of Team Build Server– Create and debug your scripts– Command line
D t d th “ t l t t” tiDoes not do the “get latest” operationmsbuild TFSBuild.proj /p:SolutionRoot=“path_to_solution_directory“– /p – specify property values (separated by semi-
colon)– /p:RunTest=false; SolutionRoot=
“path_to_solution_directory“
Questions?Questions?
AgendaAgenda
TFS & VSTS OverviewUnit Testing & Test-Driven DevelopmentTeam ProjectsSource ControlSou ce Co t oDatabase Development with VSTSTFS Build System
About the speakerAbout the speaker
Owner, Benjamin Day Consulting, Inc.– Email: [email protected]– Web: http://www.benday.comp // y– Blog: http://blog.benday.com
Trainer, Richard Hale Shaw Group– http://www.richardhaleshawgroup.com– Visual Studio Team System, Team Foundation Server
Microsoft MVP for C#Leader of Beantown.NET INETA User Group