Test Environment Management

www.kanoah.com

ABOUT US

Kanoah is an innovative company providing ground-breaking solutions to software testing professionals on the

Atlassian JIRA platform

About Kanoah Tests

Kanoah Tests is a full featured test management, integrated seamless into JIRA with the same look-n-feel. No need to learn or switch between different applications

Coordinate all test management activities including test planning, authoring, execution, tracking and reporting from a central location

Kanoah Tests enables you to track testing progress and quality to foster collaboration and visibility across traditional and agile teams

Get real-time insights into your testing progress with out of the box reports

Easily integrate your automated tests and submit test results with Kanoah Tests’ powerful REST API or use the API to automate many areas of the application

After looking for several years at plugins for test management we finally found Kanoah Tests. The other solutions were either too complex, didn't integrate well with Jira, or were focused on a single project. Kanoah Tests proved to be an elegant solution that allowed linking between any project. Kanoah has been very responsive to feedback, requests, suggestions as well bugs. The customer service is awesome. I'd highly recommend Kanoah Tests to teams of any size looking to simplify test management and consolidate tools.

Don Pierce

Robert Murhamer

Liked Kanoah from the moment I discovered it. Integrates nicely with JIRA and especially with Agile. Test cases can be authored right from the story level, but has all other functionality a Test Case Management solution would need to have, ranging from creating test plans, executing test cases, importing test cases, API for automation, a.s.o. Additionally the team at Kanoah is amazing and responding to any question very quickly. It wasn't hard to sell to my management to purchase Kanoah. Will highly recommend Kanoah to anybody.

Zour Brosh

I just start working with Kanoah and I am impressed how it's simple to manage tests without unlimited non-used features like in most of the test management tools and still to get the needed functionality and results. The integration with Jira is a great working solution that enable to share testing and development in simple way on one system. I recommend to use Kanoah for testing management. It will help to do Kanoah as much as possible customizable like Jira to match each group methodology

For more reviews, visit: https://marketplace.atlassian.com/plugins/com.kanoah.test-manager/server/reviews

Reviews

Key Features

Native seamless integration with JIRA

No need to learn or switch between different applications

Perfect for agile & traditional testing approaches

Manage, organize and track all your testing efforts in a central place

Reuse test cases across your projects

Powerful REST API

Establish clear traceability between requirements, test cases, and defects

Execute test cases and track results that matter

Get real-time insights into your testing progress with out of the box reports

Live statistics accessible to your entire team

Benefits for the testers

No need to lear or switch between different applications

Reuse test cases across projects for regression

Link test cases to requirements and defects

API support for automated efforts

Benefits for the teams

Informed decisions based on real-time insights

End-to-end traceability and impact analysis

Centralized Test Management

Save time and increase productivity

Why choose Kanoah Tests

Coordinate all test management activities right inside JIRA

1 Testing right inside JIRA

Take advantage of the built-in reports to track the results and measure progress

2Make informed decisions

Kanoah Tests users receive priority support, even during trials

3Stelar support

Test Environment Management

What is Test Environment Management?Test Environment Management (TEM) is a function in the software delivery process which aids the software testing cycle by providing a validated, stable and usable test-environment to execute the test scenarios or replicate bugs. [1]

Test environment management is the process by which test machines (clients, servers, or both) are managed. It’s super-critical to have clean machines when running automated and manual tests.

Test environments have always played a major role in software development. With a wider range of operating systems, hardware and software options available to consumers than ever before, maintaining a stable and accurate environment is becoming even more critical.

the biggest constraint in virtualized test labs is hard disk space; most organizations find they need to invest either in a SAN or some very serious RAID array technologies. [2]

What is Test Environment Management?

While test environment management will always require a certain amount of overhead, adopting a few simple practices can help you streamline the process while simultaneously increasing the quality and stability of your test environment. [3]

Maintaining a central repository of test-environments in scope with their latest version and connectivity details (Information management).

Allocation of test environments to teams as per requirement (Demand management).

Creation of new test environments as per requirement (Supply management).

Environment Monitoring (Monitoring). [1]

Activities performed by TEM

Deleting/ updating outdated test-environments and its details (Housekeeping).

Preliminary investigation of issues on the environment and sometimes co-ordination till an issue resolution (Incident Management).

Analyzing data for environment issues, identifying trends and taking pro-active steps to resolve issues / co-coordinating for a long term fix (Problem Management). [1]

Disadvantages of VirtualizationImpossibility of emulation of all devices. At the current moment, all main devices of hardware platforms are supported by vendors of virtualization systems. But if some controllers or devices, which are not supported, are used, we have to refuse the virtualization of such environment.

Virtual test environment requires additional hardware resources. Existing virtualization technologies make it possible to bring performance of virtual machines to real. But to start some definite configuration with a number of machines, we need a physical host to allow this.

Equipment conflict. Drivers used by a virtualization system can conflict with drivers of tested equipment.

In case of low disk space on the hard disk of the virtual machine, we cannot increase it if the virtual machine contains snapshots.

Final testing has to be performed on real machines because no virtual environment can completely replace the real one.

There are training costs on working with virtual machines. [4]

Disadvantages of Virtualization

When a virus is detected on one of snapshots, it is hard to define on which stage it appeared. We have to check all snapshots because the antivirus that can detect and delete viruses on all snapshots of the virtual machine isn’t developed yet.

Single destruction point. If a virtualization system fails, it is impossible to start virtual machines till the reason of failure is found and system is recovered.

It is not recommended to apply virtual systems while testing the performance and also while testing applications with high requirements to the physical resources of the computer.

The final testing of the developed software must be performed on real machines. [4]

Commonly faced challenges

1. Remote environment: The test assets or environments are mostly placed geographically in sites that are remote to the teams. This is one of the most commonly faced challenges for test teams as in case of any problems that may arise pertaining to hardware, firmware, software, networking, etc.

2. Combined usage between teams: Most often than not, development and test teams use the same environment assets. It becomes extremely cost unfriendly for organizations to procure separate resources for each team.

3. Ineffective planning for resource usage for integration: Ineffective planning with respect to usage is a large contributor of environment turning instable, besides conflict between teams. The most evident effect of this is that an issue that is noticed for a particular once or twice may produce a completely different behavior in the following runs for the same scenario. [3]

Commonly faced challenges4. Complex test configuration: The test bed or test environment configuration is sometimes too complex. This will pose several challenges as the test team will need the required skills to understand the needed configurations. In such cases the testers may themselves induce an error in the test-bed by configuring it incorrectly. This would greatly impact the test case and results it produces.

5. Elaborate setup time: In certain other times, for each test case the test setup may be far too elaborate each test case identified. This could be due to a wide variety of co-existent technologies that need to be coupled together or multiple components to work together in cases of integration testing. In these cases, each of the components have to be working perfectly to ensure consistent results, as one component may form an input to the next. [3]

Best Practices from TEMAnalyze these questions before getting started:

Do you need to be cross-platform, or do you support/test only on one platform?

Do you require complete automation, or can some manual intervention be tolerated?

How many virtualized machines do you need, and what are the required hardware specs? Can you run the automation you need in this environment?

Best Practices from TEM

The following practices have been compiled from several sources [3] [5] [6] [7].

Capture your Requirements

When a specifications document or a use case document is rolled out by the development team, the invariable step for the test team is to understand the line item requirements and then prepare a test case document detailing the test cases.

While the test planning is being carried out, it is the best practice to also include the detailed test environment information in the test case document.

Assign Test Environment Management Ownership

The first step is establishing a healthy test environment management policy is assigning ownership of the environment to an individual, team, or department.

Sharing ownership across an organization or multiple teams typically leads to accountability concerns. The result is a test environment that is poorly managed and irregularly maintained. Having a single manager or management group eliminates these problems.

Best Practices from TEMVirtualize Your Test Environment

Virtualization technology allows testers and developers to create custom instances or choose from existing template images dynamically.

Virtualization offers is the ability to take ‘snapshots’ of the test environment. These snapshots contain configuration information and are extremely useful in helping testers and developers recreate bugs.

Test Environment Assets Utilization

Consolidating environment estate is an exercise which consists of working information of all assets, owners, contact details, vendor details, physical location details, etc. This is a tedious exercise and till date one of the biggest challenges at the enterprise level.

After consolidating the assets, the organization will have to focus on creating application constellation based on patterns and should know how to simplify the test environment landscape. Sharing test environments and ensuring that resources, skills knowledge and assets are reused will minimize the operations cost.

Best Practices from TEMRegularly Reassess Your Test Environment

Get into the habit of reassessing your test environment every major test cycle. Determine what is working, what is not working, what is being underutilized, and what is being over utilized.

Keeping up with your test environment helps to ensure that it remains as accurate and realistic as possible. Assigning ownership, maintaining detailed documentation, and performing regular reassessments are three simple ways to streamline test environment management.

Regression Testing

The test environments for regression testing seem to be running on the same test setup with the same data, in actuality they are constantly evolving every release in accordance to the features being implemented as well.

Every product release cycle would have one or more rounds of regression testing. Thus establishing regression test environments for every product release cycle and re-using them within the cycle, would definitely portray the stability of the test environment.

Best Practices from TEMAutomation

Developing automation frameworks and using automation for regressive tests, also helps in improving the efficiency of a test environment because automation will assume that the environment is stable and the defects that are originated are purely feature/code oriented.

Automation strategies like Cloud technology and Service Virtualization allows the organization can fulfil ad-hoc environment requests and ensure ‘on–time’ test environment availability with lesser cost and effort.

References[1] https://en.wikipedia.org/wiki/Test_Environment_Management

[2] http://searchsoftwarequality.techtarget.com/answer/What-is-test-environment-management

[3] https://leantesting.com/resources/test-environment-management-best-practices/

[4] https://www.apriorit.com/qa-blog/223-virtualization-in-testing

[5] http://www.computerworlduk.com/infrastructure/testing-environment-management-challenges-with-virtualisation-1690/

[6] http://www.softwaretestinghelp.com/test-bed-test-environment-management-best-practices/

[7] Critical Success Factors for a Successfull Test Environment Management by Wipro.

Next Steps

Getting Started Guide

GETTING STARTED RESOURCES FREE TRIAL

DocumentationSupport

TutorialsWhat we're reading this weekBlog

Start you 30-day free trial now