Are you fully testing your code?

So what % of your code are you actually testing?

So what % of your code are you actually testing? This is a good metric for

how much of your code is tested.

Not sure? Go find someone on your team who would know. We will wait right here for you.

What did they say? Do you have 100% code coverage?

What did they say? Do you have 100% code coverage?

Hmmm…I doubt it!

We’ve heard a lot of what folks say over the years. Let’s take a

look at what they say…

How much of your code are you testing?

I don’t know how much of our code is tested…

How much of your code are you testing?

I don’t know how much of our code is tested…

Don’t worry, you’re not alone! Many companies don’t know how much of their code is tested.

Chances are though, if you don’t know, you don’t have much code coverage.

How much of your code are you testing?

I think we have around 80% code coverage!

How much of your code are you testing?

I think we have around 80% code coverage!

I hate to say it, but you probably don’t. We have had clients who thought they 80% code coverage when they actually only had 20% code coverage!

How much of your code are you testing?

I don’t know…but we have a lot of automated testing!

How much of your code are you testing?

I don’t know…but we have a lot of automated testing!

Automated testing is important but you are probably automating black box functional testing…not white box testing. No matter how much automation you have you are only getting at most 20% code coverage.

Now you’re probably wondering what’s the difference between white box and black box

testing. Let’s take a look.

What is black box testing?• This is also known as functional testing.

• Testers input values into the application and compare the output to the expected result. If the outputs do not match the expected results there is an error.

• Black box testing is only able to provide test coverage for 10% to at most 20% of an application’s source code. This is how most companies test their application.

What is white box testing?• This is also known as unit and integration testing.

• These are tests written by software developers and quality engineers. WBT examines the internal structure, logic, and syntax of an application

• It’s a sophisticated and comprehensive approach for finding defects that also streamlines the process of determining root cause for bugs.

• White box tests can test up to 90% of your source code but the drawback is they are often complicated and costly to write.

What is white box testing?• This is also known as unit and integration testing.

• These are tests written by software developers and quality engineers. WBT examines the internal structure, logic, and syntax of an application

• It’s a sophisticated and comprehensive approach for finding defects that also streamlines the process of determining root cause for bugs.

• White box tests can test up to 90% of your source code but the drawback is they are often complicated and costly to write.

Since white box testing is challenging not many software testing providers can offer it to their clients. Although an external provider can do a great job black box testing your application, you still need to find a way to white box test your application.

Let’s take a look at some bugs that could have been prevented with white box testing…

Let’s take a look at a few of those logic bugs…

A software bug could cause Boeing 787s to lose control in mid air.

A programming error costs Commonwealth Bank more than $2 million.

Mount Sinai Roosevelt erroneously submitted claims to Medicaid for payment due to a software error and ended up having to pay $2.9M to settle a false claims case.

A glitch in the FSU housing software caused the system to email approximately 13,195 current and former FSU students detailed incident reports of any and all code of conduct complaints associated with them.

Click on the speech bubbles for a link to the article about the incident.

Those are just a few examples of some pretty bad logic bugs!

The same type of bugs could happen to any application passing financial, operating, or customer data between code modules.

If you are only testing 20% of your code you are leaving 80% of it vulnerable to catastrophic bugs.

If you are only testing 20% of your code you are leaving 80% of it vulnerable to catastrophic bugs.

Remember, the 80% that is covered by white box testing which examines the internal structure and logic of the

source code. If you don’t test the logic of your application really bad bugs

can show up!

So why not just keep testing 20% of your code and fix bad bugs as they show up in production?

What’s the worst that could happen?

You’re actually spending a lot of unnecessary money fixing bugs in production instead of in development.

Let’s take a look at why.

Let’s take a look at the steps needed to fix a bug once it hits production:

1. Find out where in the code the bug is occurring2. Determine the damage caused (e.g. database corruption)3. Write scripts to correct the damage4. Write code to fix the bug5. Write code to test the bug fix6. Copy the production database to staging7. Run correction scripts on staging 8. Run scripts to check the results on the staging database9. Push code fixes to production10. Run correction scripts on production11. Run scripts to check the result on the production database

Well it looks like fixing a

production bug could be

amazingly time-consuming for an

engineer.

Why is this bad for you?

The burden is on your wallet.

Let’s take a look at the production bug cost formula:

Math!? Ugh.

(# of people X # of hours) X cost per person-hour X (# of fixed bugs) = Cost of production bug fixes

The cost of a simple bug:

Okay, $500…that might be not good but it’s not the end of the

world.

(1 person X 10 hrs) X $50/hr X (1 bug) = $500

The cost of an average bug:

$3000!? I’m sure you could think of a lot of other things your company could do with $3,000 instead of fixing one bug.

(2 person X 30hrs) X $50/hr X (1 bug) = $3000

The cost of a bad bug:

(4 person X 30hrs) X $50/hr X (1 bug) = $6000

$6,000!? That kind of money could go to happy hours, bonuses, a ping-pong

table…there are a lot more exciting things to spend money on than 1 bug.

The sad thing is this formula does not take into account two things:

1. The opportunity cost of having your engineers fixing bugs instead of writing new features.

2. The cost of brand damage due to bad bugs.

The sad thing is this formula does not take into account two things:

1. The opportunity cost of having your engineers fixing bugs instead of writing new features.

2. The cost of brand damage due to bad bugs.

So it’s expensive and hard to write white box tests and it’s expensive and

hard to fix bugs in production. Can anyone save us from this madness!?

This is where we come in.

Gasp!

GenRocket’s Automated White Box Testing Service• With our patented technology we enable engineers to create white box tests

10x faster.

• We train our quality engineers to use our technology and a give them a process on how to achieve full code coverage.

• This allows our testing service to provide your application code coverage at a level that was not previously possible from your internal testing team or external testing providers.

GenRocket’s Automated White Box Testing Service• With our patented technology we enable engineers to create white box tests

10x faster.

• We train our quality engineers to use our technology and a give them a process on how to achieve full code coverage.

• This allows our testing service to provide your application code coverage at a level that was not previously possible from your internal testing team or external testing providers.

Sounds good…must be expensive though…

Half Time Quality Engineer Full Time Quality Engineer

$3,000/month $5,000/month

20 hours/week 40 hours/week

Automated White Box Testing Service

$3,000/month – you pay that every time an average bug is

shipped to production!

Half Time Quality Engineer Full Time Quality Engineer

$3,000/month $5,000/month

20 hours/week 40 hours/week

Automated White Box Testing Service

Interested? Here’s how it works.

1. Intro CallWe will set up a one hour introduction call to get a high level understanding of your application and codebase.

2. NDAThe GenRocket Quality Engineer will be working on your codebase so we will put an NDA in place.

3. Free Code Evaluation One of our architects will sit down with one of your engineers and walk through a sample of your source code.

4. Master Service AgreementPrior to starting to deliver testing services we will put in place a master services agreement (MSA) with specific statements of work.

6. Monthly Code Coverage ReportAt the end of each month we will send you a code coverage report on how much of your codebase is tested.

5. Our Quality Engineer(s) Test Your Source CodeOur quality engineer will create white box tests starting at the top of your data model.

Sign up for a free code evaluation today.