Testing and QAon Development stack with Agile

methodologies

BINH NGUYEN THAIEtvs Lornd University,

Department of Cartography and GeoinformaticsJuly 3, 2014

1 Agile methodologies from testing and QA perspec-tive

Lets discuss briefly what is the problem with traditional waterfall devel-opment methodology? The biggest disadvantage is it assumes that everyrequirement are identified before any coding and design occurs, involvingTest Design phases:

1. Identify test conditions and test cases.

2. Design test cases.

3. Test strategies.

4. Risk evaluation.

5. Incident management.

By disintegrating waterfall model into smaller and shorter interations, weare capable of react flexibly and gradually as user requirements are clarifiedas we develop.

Agile software development methodology is all about releasing newfunctionalities in smaller cycles based on small and highly efficient team(s)which are self-organized. A sprint can be considered as a cycle or iteration,usually takes 1 to 6 weeks. Each sprint has its task, after a sprint havebeen design, its invulnerable, meaning that task can not be thrown oradded during a sprint. A sprint involves all the software development steps

1

and components including testing and QA. Agile methodologies shall beintegrated into a companys circulation gradually, meaning features anddiciplines are developed and acquired in small steps.

However, can we answer these questions before starting to implementAgile methodology with testing and QA:

Does testing and QA will effect cost values in prototype development?During prototype development, a software may contains defects andits acceptable during a demo, because our primary task is to convincestakeholders of our capabilities and prove that the concept is good.Nevertheless, do we need testing and QA at this stage? Can we makea prototype software with reusable components? How will testingand QA increase development costs?

What is the learning curve and costs for testing and QA?For an un-experienced team, with lack of testing and QA knowledge,the learning curve could take time and efforts, does Agile method-ologies gives any best practices for that? Agile manifesto only givescorner stones for development and management issues, to answer thisquestion precisely is a challenge for every organization. It dependson the team, project and development stack providing handy toolssupporting testing and QA.

Do we need technical and/or lead developers familiar with QA?We may assume that a self-organized and disciplined team have al-ready posseses testing and QA knowledge. What happens with ateam having a lead developer or technical leader, the only one personfamiliar with testing and QA, how much will it effect on quality valuesof the software, leads us to the seond question.

Do we need testing and QA across development stack?Lets take an MVC architecture providing all the tools for testing all 3layers, but what happens with the rest? Integration test, compatibilitytest, communication test across services residing on different platformsand so on. Can we guarantee our test coverage and say that its bulletproof? Can we consider our development stack as an isolated entitywithin a large software eco-system?

Tools supporting QA?Your code is stored in a code versioning system (GIT, SVN, Perforce, etc.),with classical branch, tags layout. Unfortunately most of us think of itas a file storage with versioning capability. A good repository layoutcould save time and costs. A good repository layout is a quality factor.Along with versioning system, we also need a continous integration

2

system where all the automated tests runs on every commit, notifyresposible people if there is any failure, break, error.

Opensource your software components will increase riskOpensource is a wonderfull fenomena, but one of its drawback isquality. Many people are working on the same code-base. Thesedevelopers are taking measures to implement their tests first then writetheir code using TDD to increase code coverage, however they testare usually stuck at unit level, defects commonly occur at componentlevel. How to prevent this?

We may conclude that to achieve a well-tested and high quality soft-ware we must know the capabilities of our development stack, other toolssupporting QA described in the following sections.

2 What is a Development Stack

The term is quite familiar with developers, however problems and qualityaspects become vital when your are working with more than two peo-ple. Therefore, you must get familiar with the building blocks of yourdevelopment stack. We will particularly take a closer look at web-baseddevelopment stacks based on MVC architecture.

Unlike programming and development methodologies, to get familiarwith a development stack is quite a challange. It could be complex, difficultto setup, with long learning curve. Lets take a look at each building blockor component individually:

2.1 Server-side language

There are many web-frameworks based on both object oriented and script-ing languages. Among object oriented languages JAVA is the most well-known and used. The reason is fairly simple, because the language itselffollow clearly stated standards and technologies, not to mention many bigcompanies like Google, Oracle have committed themselves with it like Play-framework or Oracle JEE and lets not forget Spring-framework. However,in order to develop a web application in Java may take some time and lotof efforts, which is not affordable in some cases, hence web-frameworksbased script languages are more suitable for startup solutions and prototypeapplications like Ruby on Rails, eg.: Twitter was originally written in Rubyon Rails, later ported to JVM. There are some interesting developmentsgoing on mixing the flexibility of scripting languages with JVM architecturelike Groovy or design a new functional language like Scala. The above

3

mentioned languages and related frameworks are multiplatform, but letsnot forget Microsoft out of the survey, who has improved a lot recently ontheir Web-framework (MVC 5), unfortunately its not multiplatform. Ofcourse, there is a work-around called Mono, but its still behind with a fewversions of .NET framework.

All of the frameworks support testing and QA support for a developerin most of the cases. We must find a close-to-perfect combination of tools toachieve the best quality of our software, which will be discussed in the nextchapters.

2.2 Database engine

A web application is simply a set of HTML pages containing information init. These information are published, modified or deleted which is stored ina database. The number of databases to choose from is vast, depending onthe nature of data, usage of information, structure of data storage we usedifferent kind of databases:

1. Graph database: Neo4J

2. Document database: MongoDB, CouchDB

3. Relational database: PostgreSQL, Oracle, MSSQL, DB2

4. Text based database: Hadoop HDFS, Google Big Table

The most important quality factor when working with databases is: con-sistency on both structural (schema) and tupple (content) levels. How toachieve and maintain it? The solution could be setting up rules and qualityaspects for both input, output and intermediate data. As for structural in-tegrity schema migration feature must be supported by development stackon database engines.

2.3 Web-Server and load balancer

The web server is responsible for serving your web-application, behindit there is usually a load-balancer, who is responsible for dispatching re-quest among application instances located on clusters. At first glance, itsa simple architecture design, but things may get complicated when a web-application is communicating with other applications or services, this iswhere integration and component testing is vital.

Getting familiar with development stack, we know where to test, butwhat tools and development methodologies exists for improving our testcoverage and quality of our software?

4

Figure 1: Development Stack

3 TDD and BDD

3.1 Test Driven Development

At first we might think of it as a software development methodology wheretests that drive your development flow. However TDD is much more thanthat, having 5 steps:

1. Developer must write tests before coding.

2. Developer runs those tests, obviously with failure.

3. Developer actually implements the code related to test cases.

4. Developer run those tests again, until all the tests have passed.

5. Developer can send the code for review, refactor, optimization.

3.2 Behavior Driven Development

BDD is a software development process based on TDD focusing on thebehavioral specification of software components. Test in BDD are designedto specify a desired behavior of the current software module. Usually abehavior specification contains:

Title Narrative, a shortly introduce who is the actor, what are the features or

business values.

Acceptance criteria for each scenario.

5

For example:

Feature: AdditionIn order to avoid silly mistakesI want to be told the sum of two numbers

Scenario: Add two numbersGiven I have entered 50And I have entered 20When execute AdditionThen the result should be 70

A step definition code is needed, for example:

Given I have entered (.*) do |n|adder = Adder.newadder.push(n.to_i)

end

execute Addition doadder.execute

end

Its your choice which you method do you prefer. Tests itself does notimprove our code as well as tests does not find bugs for you. However adeveloper uses TDD, must think, before coding. A developer must think,how a function, a class, a module and a component should work, what sideaffects (state transformations) will it make and so on. All these thoughts areimplemented in test cases. TDD is a safety belt for your software.

4 Mock-Objects

Mock objects are simply simulated objects, simulating the behavior of realobjects in a controlled manner. Mock objects are created by developers totest certain behaviors of an object. When do we need them:

1. Slow runs.

2. Difficult to produce.

3. Producing non-deterministic results.

4. Does not yet exists.

5. Intended only for testing purposes.

6

By using Mock objects, developers are focusing on testing the behavior ofthe system, without worrying about its dependencies. Most of the webapplication heavily rely on database communications, which is expensive,because tests must run fast using mock objects could reduce communicationcosts drastically.

5 Continous Integration

A continous integration system (CI) are systems that build, test softwareautomatically and generate report. They can run slow, data-intensive testsor verification of proper performance test on different platforms. Users aregetting immediate feedback of current or previous status and stages of thesoftware.

The primary advantage of using a CI system from QA aspects is im-mediate feedback on any defect. Both developer team and testing team canreact to this, localizing the source of that defect is fairly simple, because onthe report, we know, which modules test case(s) have been broken. Alsometrics are generated from automated testing like:

1. code coverage

2. code complexity

3. feature completeness

Continous integration itself does not give you quality at all. The core ofa good continous integration system is the work-flow that developer teamhas set-up for themselves. A good layout of code repository is also veryimportant, for example product owners request the current version of thesoftware is quite simple: we take the latest stable version of the softwareand deliver it, however when product owners wants a version with certainset of features included under-development and bug-fixed functionalities,we may have a hard time to produce it, CI is useless if the repository layoutis not configured properly.

6 Conclusion

Despite the knowledge of software testing and QA there are many othermethodologies, architectures, techniques and tools we must get familiarwith especially in a rapidly changing world of Agile development. Testingand QA questions and answers on a development stack itself is crucialto fullfill a good quality software. QA engineers, disciplined team andmanagement is required.

7