************ [Introduction to Test Driven Development (TDD)]********************************************

Test-driven development is an evolutionary approach to development which combines test-first development where

you write a test before you write just enough production code to fulfill that test and refactoring.

The steps of test first development (TFD): The first step is to quickly add a test,

basically just enough code to fail. Next you run your tests, often the complete

test suite although for sake of speed you may decide to run only a subset, to

ensure that the new test does in fact fail. You then update your functional code

to make it pass the new tests. The fourth step is to run your tests again. If they

fail you need to update your functional code and retest. Once the tests pass the

next step is to start over (you may first need to refactor any duplication out of

your design as needed, turning TFD into TDD).

TDD = Refactoring + TFD.

Instead of writing functional code first and then your testing code as an

afterthought, if you write it at all, you instead write your test code before your

functional code. Furthermore, you do so in very small steps – one test and a

small bit of corresponding functional code at a time. One of the advantages

of pair programming is that your pair helps you to stay on track.

Two levels of TDD

Acceptance TDD (ATDD). With ATDD you write a single acceptance

test, or behavioral specification depending on your preferred

terminology, and then just enough production functionality/code to

fulfill that test. The goal of ATDD is to specify detailed, executable

requirements for your solution on a just in time (JIT) basis. ATDD is also called Behavior Driven Development

(BDD).

Developer TDD. With developer TDD you write a single developer test, sometimes inaccurately referred to as

a unit test, and then just enough production code to fulfill that test. The goal of developer TDD is to specify a

detailed, executable design for your solution on a JIT basis. Developer TDD is often simply called TDD.

Testing via the xUnit Framework.

Beck’s 2 rules for TDD: 1. you should write new business code only when an automated test has failed 2. you should eliminate any duplication that

you find

Good unit tests:

Run fast (they have short setups, run times, and break downs). Run in isolation (you should be able to reorder them). Use data that makes them easy to read and to understand. Use real data (e.g. copies of production data) when they need to. Represent one step towards your overall goal.

TDD increases your confidence that your system actually meets the requirements defined for it, that your system actually

works and therefore you can proceed with confidence.

An interesting side effect of TDD is that you achieve 100% coverage test – every single line of code is tested – something

that traditional testing doesn’t guarantee (although it does recommend it).

Well-written unit tests provide a working specification of your functional code – and as a result unit tests effectively

become a significant portion of your technical documentation. The implication is that the expectations of the pro-

documentation crowd need to reflect this reality. Similarly, acceptance tests can form an important part of your

requirements documentation.

TDD is very good at detailed specification and validation, but not so good at thinking through bigger issues such as the

overall design, how people will use the system, or the UI design (for example). Modeling or agile model-driven

development (AMDD) is better suited for this.

The Agile Model Driven

Development (AMDD)

lifecycle.

Comparing TDD and AMDD:

TDD shortens the programming feedback loop whereas AMDD shortens the modeling feedback loop. TDD provides detailed specification (tests) whereas AMDD is better for thinking through bigger issues. TDD promotes the development of high-quality code whereas AMDD promotes high-quality communication with

your stakeholders and other developers. TDD provides concrete evidence that your software works whereas AMDD supports your team, including

stakeholders, in working toward a common understanding. TDD “speaks” to programmers whereas AMDD speaks to business analysts, stakeholders, and data professionals. TDD is provides very finely grained concrete feedback on the order of minutes whereas AMDD enables verbal

feedback on the order minutes (concrete feedback requires developers to follow the practice Prove It with Code and thus becomes dependent on non-AM techniques).

TDD helps to ensure that your design is clean by focusing on creation of operations that are callable and testable whereas AMDD provides an opportunity to think through larger design/architectural issues before you code.

TDD is non-visually oriented whereas AMDD is visually oriented. Both techniques are new to traditional developers and therefore may be threatening to them. Both techniques support evolutionary development.

Which approach should you take? The answer depends on your, and your teammates, cognitive preferences.

How do you combine the two approaches? AMDD should be used to create models with your project stakeholders to help explore their requirements and then to explore those requirements sufficiently in architectural and design models (often simple sketches). TDD should be used as a critical part of your build efforts to ensure that you develop clean, working code. The end result is that you will have a high-quality, working system that meets the actual needs of your project stakeholders.

Myths and Misconceptions

Unit tests form 100% of your design specification

TDD doesn’t scale

You only need to unit test

TDD is sufficient for testing

You create a 100% regression test suite

Test-driven development (TDD) is a development technique where you must first write a test that fails before you write

new functional code. TDD is being quickly adopted by agile software developers for development of application source

code and is even being adopted by Agile DBAs for database development. TDD should be seen as complementary to Agile

Model Driven Development (AMDD) approaches and the two can and should be used together. TDD does not replace

traditional testing, instead it defines a proven way to ensure effective unit testing. A side effect of TDD is that the resulting

tests are working examples for invoking the code, thereby providing a working specification for the code. My experience

is that TDD works incredibly well in practice and it is something that all software developers should consider adopting.

************* [The Developers Guide to Debugging] *****************************************************

Opportunities how to find Bugs

Debuggable source code

Instrumentation

Macro definitions

Compiler flags

Static checkers

Selected libraries

Linker options

Code instrumentation tools

Test case / input data

Debuggers o Source code o Profiling o Memory access o OS call tracers such as truss or strace

The 13 Golden Rules of Debugging 1. Understand the requirements 2. Make it fail 3. Simplify the test case 4. Read the right error message 5. Check the plug [check the seemingly obvious] 6. Separate facts from interpretation [don’t jump into conclusions, can you prove it? Is it a fact?] 7. Divide and conquer

Understand how to slice and dice the problem. Use your revision control system to determine the last version of the source code that did no misbehave, by checking out complete configurations with different time tags and building them. Similarly, determine the first version that exhibits the problem. Then analyze the source code changes between these two versions – your bug is hiding there.

May have to face the tedious task of side-by-side debugging comparing data, log files and the flow of control in both versions, concurrently, side-by-side. 8. Match the tool to the bug 9. One change at a time 10. Keep an audit trail [Keep track of your changes] 11. Get a fresh view 12. If you didn’t fix it, it ain’t fixed 13. Cover your bug fix with a regression test

Build a Good Toolkit

• Have the following installed and tested with the software you are developing (a 10-line test program does not count):

– A source code debugger – A memory debugger – A profiler

• Ensure that your debuggers are compatible with your compiler • Run and maintain unit and system tests • Automate frequent tasks

Running tests every day keeps the bugs at bay > these tests should be automated in order to allow for frequent and

efficient execution. And the tests should be self-checking. There should be a regression test system – typically a

collections of scripts – that executes a steadily growing set of tests. The outcome is a list of tests that passed and one

that shows a set of failing tests. Tests are added for each new feature and whenever a bug is fixed.

Unit Tests and System Tools

Let’s differentiate these concepts; a system test uses your software as a whole. These tests are necessary, they emulate

normal operation and ensure end-user functionality. Unit Tests on the other hand focus on the individual building blocks

of your software in isolation. Typically, they require extra effort in the form of additional test executables or test

interfaces.

We can also distinguish between white and black box unit tests. Black box tests are focusing on verifying the intended

functionality of a component, ignoring its actual implementation. The advantage of black box tests is their portability –

even if the implementation changes, these test will still work correctly. White Box tests on the other hand are focused at

testing corner cases of the implementation and your “insanely clever hacks”. They make really good watchdogs.

Bugs Classification

1. The Common Bug

2. Sporadic Bugs

3. Heisenbugs

a. The Race-ist > situations where the program behavior depends on the order of execution of certain pieces

of code, and this order is not well defined.

b. The Memory Outlaw > Memory access violations such as reading uninitialized variables, dangling pointers

or array bound violations.

c. The Idealist > Dwells in optimizations of abstract data types and algorithms, and strikes when the

corresponding code takes some sort of illegal shortcut.

4. Bugs hiding behind bugs

5. Secret Bugs – debugging and confidentiality

a. Reproduce Bugs In-House > ask for the test case or a stripped-down test case. If that is not possible due

to confidentiality or time constraints, then try to reproduce the same bug in house.

b. On-site debugging > Compile your software with debug information. Send it to your customer but leave

out all source-code files.

c. Using Secure Connections

The quickest and most efficient tool to visualize a program’s behavior is a debugger, a program to test and debug other

programs. A debugger is called a source code debugger or symbolic debugger if it can show the current point of execution,

or the locations of a program error, inside the program’s source code.

Prepare a simple predictable example > get the debugger to run with your program > learn to do a stack trace on a

program crash.

The stack of a C/C++ program is a segment of memory assigned to storing one stack frame for each active functions call. A stack frame consists of the return address, and the function’s arguments and local variables. A stack trace is the actual chain of stack frames from the topmost function where the debugger is currently stopped or paused, down to the function main(). A stack overflow occurs when the chain of nested function calls gets so long that the stack does not have enough memory to store the current stack frame. In addition to showing the location of a program crash in the source code, a debugger will also show the stack frame and stack trace of the crash. A stack trace is useful information for debugging a crash, since it tells you the chain of functions calls that led to the crash. Learn to Use Breakpoints > Learn to navigate through the program > Learn to inspect data: variables & expressions

• Line breakpoint – will pause the program when it reaches a specific line in the source code. • Function breakpoint – will pause the program when it reaches the first line of a specific function. • Conditional breakpoint – will pause the program if a certain condition holds true. • Event breakpoint – puts the program in pause mode if a certain event occurs. Supported events include signals from the operating system, and C++ exceptions.

Lessons learned:

• Use a source code debugger to visualize a program’s behavior. • Prepare a simple example to familiarize yourself with the features of the debugger. • Get the debugger to run with your program. • Learn to analyze the stack trace of a program crash. • Learn to use breakpoints. • Learn to navigate through the program. • Learn to inspect variables and expressions. • Do a debug session on a simple example.