Software Quality Sourcebook
Transcript of Software Quality Sourcebook
-
8/12/2019 Software Quality Sourcebook
1/20
http://www.sdtimes.com/ -
8/12/2019 Software Quality Sourcebook
2/20
Platinum Sponsor Gold Sponsor
Silver Sponsors
April 8-10, 2013
Boston, MA
www.BigDataTechCon.com
Discover how to master Big Data from real-world practitioners instructors
who work in the trenches and can teach you from real-world experience!
A BZ Media Event Big Data TechCon is a trademark of BZ Media LLC.
Big Data gets real
at Big Data TechCon!
Come to Big Data TechConto learn the best ways to: Collect, sort and store massive quantities of structured
and unstructured data
Process real-time data pouring into your organization
Master Big Data tools and technologies like Hadoop,
Map/Reduce, NoSQL databases, and more
Learn HOW TO integrate data-collection technologies
with analysis and business-analysis tools to produce
the kind of workable information and reports your
organization needs
Understand HOW TO leverage Big Data to help
your organization today
Untangling the Relationship Hairball with a Graph Database
Beyond MapReduce
In-Database Predictive Analytics
Microsofts Big Data Story
Hands-on NoSQL for the DBA
MapReduce Tips and Tricks
Apache Cassandra A Deep Dive
HBase Schema and Table Design Principles
A Survey of Probabilistic Data Structures
High-Speed Data Ingestion with Sharded NewSQL Databases
Implementing a Real-Time Data Platform Using HBase
Big Data Science: Extracting Truth from Large,Multi-structured Data, Parts I & II
Analytics Maturity Model
How to Fit a Petabyte in Apache HBase
Building an Impenetrable ZooKeeper
Mastering Sqoop for Data Transfer for Big Data
Building High-Volume Web Applications
Building Successful Data Science Teams ...and LOTS more!
Download the full Course Catalog TODAY!
Over50how-to
practicalclass
es
andtutorials
tochoose
from!
Check out this list of classes!
The HOW-TO conference for Big Data and IT Professionals
Register Early and SAVE!
Hyatt Regency Cambridge
Adam LaiacanoData Scientistand Engineerat Tumblr
Improve, Dont Abuse:
Use Big Data for
Your Customers,
not Against Them
Oscar BoykinStaff SoftwareEngineer atTwitter
Addition in the Large:
Simple Counts and
Not-So-Simple Counts
Dr. MichaelStonebrakerDatabase Pioneerand AdjunctProfessor at MIT
Big Data Is (at Least)
Four Different Problems
Check out these keynotes!
http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/http://www.bigdatatechcon.com/ -
8/12/2019 Software Quality Sourcebook
3/20
THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK MARCH 3
Quality as Art,
Science, NecessityQ
uality is essential in every software development project. Yet while com-
puter science programs teach developers about software architecture,
object-oriented design, algorithms and programming, theres little formal
emphasis on testing and quality assurance. Its left to managers and employers,
in many cases, to both teach the fundamentals of quality assurance and inspire
a culture of quality.
Every day, managers implore their programmers to code faster. But do they
provide them with the tools and knowledge needed to write better code? Do they
equip their test teams with the best resources to stamp out defects and vulnera-
bilities? All too often, testing is neglected at every stage. Thats not only true inenterprise development, of course, but also in small-business environments,
contract work, open-source projects and more. We
preach quality as an ethic... and then end up having to
enforce it with tools and policies.
Quality assurance is an art. A developer or tester
needs to understand where quality assurance fits into
the application development life cycle, and why theres
more to quality than passing a test suite or meeting some
arbitrary metric.
Quality assurance is a science. Over the past 30 years,
a significant body of knowledge has produced best prac-
tices for functional testing, unit testing and performance
testing, but now those classic techniques are being aug-
mented by recent innovations in the field of defect man-
agement, test automation, software-configuration management, metrics, test
development, and security/vulnerability testing.
Quality assurance is a necessity. As you will read in Software Quality: More
Than Testing, software errors cost real money, not only in wasted productivity
and product delays, but also in real business losses and even legal fines. Thats
not what you want.
This Special Supplement to SD Times explores solutions providers focused on
helping you improve software quality. All of the companies profiled in this Sup-
plement offer test and quality-assurance tools that can make a real difference.
Be sure to consider these companies when evaluating quality solutions for
your team, or for your entire organization.We hope you enjoy this special Supplement to SD Times.
ContentsFeatureSoftware Quality: More Than Testing ........................4
By Lisa Morgan
Software Quality LeadersParasoft Eliminates Trade-Offs BetweenInnovation Speed and Quality...................................7Parasoft Platinum Sponsor
Design Quality with TechExcel...............................11TechExcel Platinum Sponsor
Test/QA AutomationIncreasing Testing Productivity with IDTs
Automated Test and Re-Test (ATRT)......................15Innovative Defense Technologies Gold Sponsor
Test ManagementReal-Time Test Management
from Zephyr Drives Instant Visibility.....................17Zephyr Gold Sponsor
Mobile Test/QAMaster Software Quality with Orasi ......................19Orasi Silver Sponsor
Alan Zeichick
Editorial Director
Alan Zeichick
Senior Art Director
Mara Leonardi
Copy Editor
Adam LoBelia
Lead Writer
Lisa L. Morgan
Customer Service/
SD Times Subscriptions
+1-847-763-9692
BZ Media
President
Ted Bahr
Executive
Vice President
Alan Zeichick
BZ Media LLC
7 High Street, Suite 407
Huntington, NY 11743
+1-631-421-4158
FAX +1-631-421-4130
www.bzmedia.com
A Supplement to SD Times March 2013 2013 BZ Media LLC, All Rights Reserved
-
8/12/2019 Software Quality Sourcebook
4/20
Software development has rapidly evolved to meet the
needs of todays fast-paced business environment, but
testing and quality-assurance practices have not neces-
sarily kept pace as evidenced by high-profile failures and
less-than optimal user experiences. Although solution
providers have been addressing the development/QA gap for
a while, their approaches are becoming more sophisticated
to align with the modern requirements of a world that runs
on software.
Software has permeated almost every aspect of our lives,and it really does drive business innovation, said Wayne Ario-
la, chief strategy officer at test-solution maker Parasoft. Yet,
we as an industry have not truly addressed the cost of quality
as a business issue.
Software failures cost companies billions of dollars each
year. Real money. Last year, in a matter of minutes, Knight
Capital lost $440 million, causing its stock to plummet 75%,
which in turn triggered a class-action suit on behalf of share-
holders. That failure also caused the United States Securities
and Exchange Commission to start drafting new automated-
trading rules. Software glitches also were to blame for the
delay of Facebooks IPO on its first tradingday, as well as for canceled United Air-
lines flights. Whether the problem is
caused by an errant algorithm or the
merging of two companies mission-criti-
cal applications, inadequate testing is
usually blamed.
Failures often stem from a lack of
established standards for software release
quality, said Tieren Zhou, founder and
CEO of TechExcel, a test-solutions
provider. Many times, quality standards
are not established because it is difficult toobjectively quantify quality.
While metrics such as the number of
bugs and the amount of test coverage have
historically served as indicators of soft-
ware quality, now one must consider the
complex operating environment, the user
experience and the risks to the business.
As an example, California sued Delta air-
lines for failing to include a privacy policy
in its Fly Delta mobile app.
There were probably some very exu-
berant developers working on the Deltamobile app who pushed it out to the marketplace as
fast as possible without understanding the business implica-
tions of doing so, said Parasofts Ariola.
Speed KillsMany organizations have embraced agile practices with the
goal of releasing smaller pieces of software faster. At the same
time, the nature of software and the environments in which it
runs have become more complex. Balancing the needs for
both speed and quality can be challenging as software ratings
and reviews indicate.The world is operating from a quality perspective the same
way its always operated: It essentially views tests as something
that happens at the end of a cycle, and so errors are discovered
really late, said Ariola.
While testing earlier in the software development life cycle
isnt a new concept, the notion of what should be tested earli-
er is changing. Parasoft and TechExcel are good examples.
Parasoft has a virtual test environment developers can use
to more effectively prevent errors from appearing in actual test
environments. It allows developers to do functional testing
and performance testing in addition to the usual unit testing
and static analysis. Parasoft also has a development-test-ing platform that enforces quality processes, including
static analysis, unit testing, coverage analysis, code
reviews and runtime testing, so software teams can ensure that
quality-assurance standards are being met consistently.
TechExcels DevTest test-management and quality-assur-
ance solution also enforces quality standards. It integrates
with defect-tracking tools and allows test cases
to be tied to requirements, so quality can be
built into products earlier in the software life
cycle.
You need to quantify quality in a stan-
dard way that can be applied to both agile andnon-agile teams, said TechExcels Zhou.
Quality control should begin as soon as the
first requirement is committed. Once you
know what features youre going to build, you
should be thinking about how they should be
tested so they can deliver a high-quality expe-
rience to the end users.
Better Ways to AutomateEnterprises and ISVs have been trying to address
the need for speed and more comprehensive test-
ing using test-automation solutions. Automatedtesting allows more tests and more types of tests to
Software Quality:
More than TestingBy Lisa L. Morgan
4 MARCH THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK
-
8/12/2019 Software Quality Sourcebook
5/20
be executed faster than would be possible to do manually,
although testing more doesnt always mean testing smarter.
Developers get all this time to write the code, then theywant to shove it into a pipe where automation happens and
then clean code comes out, said Parasofts Ariola. Automa-
tion is exposing just how brittle the testing infrastructure is.
Parasoft and TechExcel are addressing the higher-level prob-
lem of having enforceable quality standards in place that apply
to both automated and manual testing. Innovative Defense
Technologies provides consulting services that help organiza-
tions apply automated testing practices in
ways that align with predefined outcomes.
IDT also offers military-grade test-manage-
ment and analysis solutions that are used by
the U.S. Department of Defense, DoD con-tractors and enterprises that are doing life-
cycle software testing in highly complex
environments.
While automated testing can accomplish
more in less time, fast release cycles can
nevertheless be negatively impacted by time-intensive status
meetings and traditional reporting practices. Zephyr is
addressing this issue with real-time test-management solu-
tions that provide 24x7 visibility into the status of tests via
dashboards.
Complex Environments Complicate TestingThe growing number of platforms, protocols, systems,
devices, networks, apps, software components and software
versions continues to complicate software testing. TechEx-
cels Zhou said that to ensure quality, test cases have to be
designed for each requirement, and that the permutations
for all the possible runtime variables need to be specified for
each test case.
Parasoft is addressing environmental complexity with sim-
ulated test-environment and cloud/SOA/API testing solutions.
Were seeing major disruptions in the SDLC today because
clouds are challenging the concepts of where software lives
and how it should behave. We also have mobile devices that areforcing us to rethink the user experience and whats impor-
tant, said Ariola. Then we have the concept of APIs, which are
driving composite applications, so testing is a more complex
endeavor.
Clouds represent three problems. First, they are forcing
software teams to expand testing outside their own walls. Sec-
ond, when services and software run in the cloud, they are sub-
ject to cloud disruptions. One high-profile example was the
Netflix outage that occurred on Christmas Eve 2012, which, as
reported by TechCrunch, was caused by errors caused by Ama-
zon Web Services Elastic Load Balancing API calls. Amazon
Web Services disruptions have also negatively affected suchhigh-traffic sites as Foursquare, Instagram, Pinterest and Red-
dit. When the failures occur, users blame the brand, not the
platform. Clouds are also lowering software-switching costs.
Many software publishers have replaced or complementedtheir on-premise solutions with SaaS-based alternatives.
Monthly SaaS subscriptions are currently popular, although
some users are clamoring for pay-per-use and shorter-term
models. The trend is extending to software testing, which is
why Orasi offers a cloud-based version of HP LoadRunner that
is available in 24-hour increments.
The lower out-of-pocket costs and easy onboarding of SaaS
solutions and mobile apps versus tradition-
al software translate to lower costs of
switching, underscoring the need for com-
prehensive software testing. Parasofts Ario-
la replaced an SaaS-based marketing-automation solution because it failed to
load critical data in a timely fashion. The
final straw was an executive presentation
in which the graphics failed to load initially,
followed by graphics that displayed stale
data. In three days, he migrated all his data assets to a new
platform and integrated the new solution with Salesforce.com.
When mobile app users get frustrated, they simply delete the
app. In the case of on-premise software, customers may simply
decide against upgrading if the switching costs are too high.
Customers lose confidence when your software quality is
poor, said TechExcels Zhou. You can try to make up for itwith a fancy marketing campaign, but if youve lost your cus-
tomers trust, theyll stick with the older versions. This cost of
maintaining old versions is a huge expense for ISVs, and it can
also hurt your future growth.
Bottom LineQuality cant wait for late-stage testing if software teams want
to ensure it. While developers wont be replacing QA engineers
anytime soon, there is merit to testing more aspects of quality
earlier in the software life cycle in a manner that doesnt derail
software development. While agile development teams often
include developers, testing/QA, and maybe end users, testingby developers is not a substitute for QA, and QA is not a substi-
tute for designing quality in to begin with.
Zhou suggested assigning QA floaters to development
teams so that more comprehensive tests can be created while
software is being developed. The alliance gives developers bet-
ter insight into quality, and QA teams get better insight into
what needs to be tested and why.
We have to remove the constraints associated with waiting
until a project is complete in order to test, said Parasofts Ari-
ola. If you really want to reduce risks, you have to [test earlier
in the cycle], compress your defect-discovery curve, and pro-
mote methods that prevent errors from getting into the soft-ware development life cycle in the first place.
THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK MARCH 5
While developers wont be
replacing QA engineers
anytime soon, there is
merit to testing more
aspects of quality earlier.
-
8/12/2019 Software Quality Sourcebook
6/20
http://sdtimes.com/pdf/TakeControl/1867 -
8/12/2019 Software Quality Sourcebook
7/20
Software quality stinks! Defect-free software is difficult to
achieve with short release cycles and the ever-increasing com-
plexity of environments. With Parasoft Service Virtualization,
Development Testing and API Testing solutions, organizations can
replace incomplete testing practices with software-quality process-
es that deliver defect-free software faster and more efficiently.
Software is the primary interface to the business. Defects or fail-
ure to satisfy end-user expectations have much greater impact as
application-switching costs are at an all-time low. Yet the approach
to software quality is the same as it was 20 years ago, said Wayne
Ariola, chief strategy officer at Parasoft. As an industry, we need to
realize that the cost-schedule-quality triangle has shifted, and wecan no longer compromise quality. Gone are the days of end users
being QA testers. Software quality must be achieved with a repeat-
able process for the business to survive.
Parasoft helps you achieve an unprecedented level of
defect prevention and process visibility with the indus-
trys first Development Testing Platform. This is
complemented by test-environment simula-
tion technology, which lets an organization test
anytime, anywhere. Using Parasofts solutions
lets teams deliver thoroughly tested software
knowing it is secure, reliable and compliant.
Test Anytime, AnywhereParasofts Service Virtualization simulates a
complete test environment so that more comprehensive testing can
be accomplished earlier in the software development life cycle.
Rather than waiting for defects to appear in actual test environ-
mentsor worse: in the fielddevelopers can simulate any envi-
ronment necessary for accurate functional testing and performance
testing, as well as agile and parallel development.
If you really want to improve software quality, you need to be
able to access a realistic, high-fidelity test environment early in the
process, said Ariola. If youre waiting until the project is complete
to test, youre discovering errors really late. Delays are rampant in
release cycles; for example, new features are added, or unexpected
complexity in the implementation. Since the release date wont
necessarily change, the first thing that gets cut is testing. Service
Virtualization gives you a simulated test environment so you can
begin testing much sooner and catch defects much earlier.
With an intuitive interface, team members can model complex,
simulated test environments, and then run large-scale, high-
throughput load and functional tests without impacting existing
systems or operations. Since a simulated test environment is creat-
ed, there is tremendous flexibility to model scenarios that would be
impossible to recreate in a staged test environment.
Reduce RisksDelivering more software features in less time means trade-offsbetween coding and testing. With Parasofts Development Testing
platform, software teams can implement and enforce processes
that reduce the risk of software failures. The platform ensures more
comprehensive static analysis, unit testing, coverage analysis, code
reviews and runtime testing, as well as more accurate traceability.
When you shift quality left, developers become inundated
with fixing defects at the same time that they are responsible for
building new features. Because a developer only has so much time,
you must promote methods that prevent errors from getting into
the software development life cycle in the first place, said Ariola.
Service Virtualization and Development Testing are now converg-
ing in a way that not only helps organizations start testing earlier,
but also reduces the amount of defects that are injected into thesoftware. This means that you can not only test earlier, you can
also actually test less and go to market significantly fasterwith
more innovative, better-quality deliverables.
Notably, Parasofts Develop-
ment Testing platform enables
organizations to implement
and enforce a defect-preven-
tion strategy that ensures best
practices. Developers can build
security into their applications,
prevent costly defects, and get
the traceability and auditability they need to comply with industry
standards. When software teams embrace policy-driven develop-ment, they can assure the reliability and performance of their
applications.
We introduced the software policy concept eight years ago in
the industrys first development testing platform. Developers
always say that theyre going to do static analysis or unit testing, but
there was no way of measuring the effectiveness of it or determin-
ing if it was being completed, said Ariola. Using our Development
Testing platform, you can declare how the code should be delivered
to satisfy business expectations. Then you can automate adherence
to the policy, aligning development with business goals.
Ensure Quality in Complex EnvironmentsParasofts API Testing solution simplifies the testing of critical APIs,
SOA, or other composite or cloud-based applications by automat-
ing complex test scenarios across Web UIs, messaging layers, enter-
prise service buses, databases, and mainframes. It enables accurate
event tracing and multi-layer validation while providing the flexi-
bility needed to easily extend, update, and reuse test artifacts.
APIs are driving composite applications, requiring organiza-
tions to orchestrate multiple endpoints. Its a complex endeavor
thats fraught with challenges, said Ariola. Our API-testing solu-
tion ensures API integrity so you can be confident that you are
releasing secure, reliable and compliant business applications.
By integrating Service Virtualization, Development Testing and
API/Cloud/SOA Testing, Parasoft reduces the time, effort and costof delivering secure, reliable and compliant software.
Learn more at www.parasoft.com.
Parasoft Eliminates Trade-Offs Between
Innovation Speed and Quality
THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK MARCH 7
Gone are the days of end usersbeing QA testers. Softwarequality must be achievedwith a repeatable process.
Wayne Ariola
SOFTWARE QUALITY LEADERSPlatinum Sponsor
-
8/12/2019 Software Quality Sourcebook
8/20
http://sdtimes.com/pdf/TakeControl/1867 -
8/12/2019 Software Quality Sourcebook
9/20
http://sdtimes.com/pdf/TakeControl/1867 -
8/12/2019 Software Quality Sourcebook
10/20
http://sdtimes.com/pdf/DevSuite9.0/1868 -
8/12/2019 Software Quality Sourcebook
11/20
Achieving consistent quality levels across all software projects is
difficult given the typical variations in software team practicesand runtime variables. With TechExcels DevTest, organiza-
tions can improve quality levels using limited QA resources, as well
as provide a means for both agile and traditional teams to integrate
QA processes into their development processes.
If you want to establish an objective standard for quality, you
have to be able to attach a quantified numeric value to it. Without
an accurate measurement of quality, it becomes harder to plan
quality at earlier stages of the software development life cycle, said
Tieren Zhou, founder and CEO of TechExcel. DevTest allows you to
repeatedly deliver quality releases.
Even though the length of software development cycles is con-
tinuing to shrink, many organizations are still using word proces-sors and spreadsheets to track software quality and test cases. But
regardless of whether testing is manual or automated, the test cas-
es often fail to cover the full range of the target softwares behavior,
resulting in errors, rework and delays.
The complexity of runtime variations makes it very
difficult to cover all the aspects of software behavior that
determine quality, said Zhou. We have
enabled quality quantification and reporting
for any release, sub-release or iteration. For
the past 12 years, we have been dedicated to
making software quality measurable.
DevTest Enforces Quality StandardsDevTest is a complete quality-assurance-
management solution that enforces testing
standards and improves efficiency across all test cycles. It allows
users to execute test assignments and submit defects using the
same interface. Users also have the ability to create a central repos-
itory for test cases, knowledge items and test automation scripts, so
quality can be achieved consistently.
TechExcel allows you to integrate quality-assurance practices
with a development project regardless of the methodology being
used, said Zhou. You can define test cases as well as the imple-
mentation process to ensure that specifications and requirements
are tested and verified during every stage of development.The DevTest system integrates with popular automated testing
tools using TechExcels TestLink API.
While testing early and often is a well-accepted practice, qual-
ity is not always contemplated at the development-task level.
Using DevTest, software teams can account for all test phases
using detailed histories of test cases, data and test results. The cen-
tralized test library also improves test standardization, reuse and
revision control.
DevTest allows you to comprehensively represent all aspects of
quality assurance by utilizing test cases that are predefined in a
template, said Zhou. The template defines the functional testing
procedures and the expected result.
DevTest allows easy scheduling of automated and manualtests, and also provides real-time test-result reporting, tracking
and analysis, so the exact state of testing is always known. And it
provides out-of-the box integration with JIRA, Mantis, Bugzillaand DevTrack, allowing easy integration with existing tracking
tools.
Compliance-management solution provider Gael Quality had
been using spreadsheets to document tests and record results
before adopting DevTest. The company was looking for a solution
that could handle more than 15,000 test cases, and could integrate
results from external sources. In just three days, the company was
able to implement DevTest and begin testing at scale. Before long,
Gael Quality was able to manage 15,000 master test cases and well
over total 100,000 test cases with ease, which was previously impos-
sible using manual methods.
Build Quality into DevelopmentMany TechExcel customers use a combination of DevTest and Dev-
Track defect tracking to improve the results of testing in develop-
ment and overall quality levels. As tests are executed, defects are
automatically flagged so issues
can be resolved efficiently.
DevTrack automatically routes
issues to the appropriate user
when an issue has been creat-
ed or forwarded. If the issue is
not resolved within the user-
specified time frame, it is auto-
matically escalated to notify the appropriate engineers or issueowners.
DevTrack provides a quality-feedback mechanism, and it helps
build quality into software during software development, said
Zhou. As developers start to work on development tasks, test cases
are presented as a special tab. Meaning, for every single task, a test
case can be defined within the developers process.
DevTrack supports sophisticated workflow rules for controlling
issue states and status transitions. It also allows subtasks to be cre-
ated and assigned to team members so they can help track and
resolve parent work items and issues. If the status changes on a par-
ent issue, pre-defined sub-issues can be triggered automatically.
In addition to providing full access control, DevTrack can be
integrated with popular SCM tools, so source-code files and
check-in/check-out operations can be tied to DevTrack work
items and defects.
Ensure Complete TraceabilityDevTest and DevTrack seamlessly combine with DevSpec, TechEx-
cels requirements-management solution, enabling complete, end-
to-end traceability. The products are available individually or
together as DevSuite, a complete ALM solution.
If you want to ensure quality, test cases should be linked with the
requirements so you have total traceability driving quality, said Zhou.
When the test cases associated with every single development task
are verified, you can dramatically improve software quality.For more information visit www.techexcel.com.
Design Quality with TechExcel
THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK MARCH 11
Without an accurate measurementof quality, it becomes harderto plan quality at earlierstages of the dev cycle.
Tieren Zhou
SOFTWARE QUALITY LEADERSPlatinum Sponsor
-
8/12/2019 Software Quality Sourcebook
12/20
http://sdtimes.com/pdf/DevSuite9.0/1868 -
8/12/2019 Software Quality Sourcebook
13/20
http://sdtimes.com/pdf/DevSuite9.0/1868 -
8/12/2019 Software Quality Sourcebook
14/20
http://sdtimes.com/pdf/Authority/1869 -
8/12/2019 Software Quality Sourcebook
15/20
Test organizations are increasingly challenged with needing to
test larger, more complex software baselines in less time, while
also being able to ensure a high-quality software product is
released. Innovative Defense Technologies (IDT) has developed an
automated testing suite of tools called Automated Test and Re-Test
(ATRT) to address these testing challenges. With the use of IDTs
ATRT suite of tools, enterprises with mission-critical systems, such
as the U.S. Department of Defense (DoD), can increase the volume,
test coverage, and complexity of integration and accept-
ance tests while dramatically reducing test cycle times.
In todays environment, software teams want to get
to market fast at the lowest cost without compro-
mising software quality, said Bernie Gauf, presi-
dent and CEO of IDT. ATRT is an innovative tech-
nology that we have consistently seen improve
testing productivity by more than 75%.
Since 2006, IDT has been pioneering new
ways of designing, developing and implementing
best-in-class automated software testing solu-
tions for the DoDs complex systems. The compa-
ny also offers automated test-strategy and design services that
improve the effectiveness of testing and QA programs.
Automate Tests with Confidence
IDT offers two robust DoD-approved ATRT solutions that supportthe entire testing life cycle: ATRT Test Manager (TM) and ATRT
Analysis Manager (AM).
ATRT TM is designed to support the testing of large, complex,
heterogeneous multicomputer systems. ATRT TM is non-intrusive
to the system under test, and provides a model-based approach to
GUI automation, which does not require the development of any
test scripts.
Another thing that differentiates ATRT TM is its ability to evalu-
ate all messages at the message-traffic level. Its message-based
interface testing capability spans multiple protocols, including
SOAP over HTTP, TCP/IP, UDP and Multicast.
As software products get more complex and depend on interac-
tion with other software products, the interfaces between compo-
nents become critical, said Gauf. You have to be able to validate
the performance of software products at the interface-specification
level to have high confidence that once systems are deployed, they
will work as expected.
ATRT TM also allows you to combine GUI-based automation
with message-based automation in one test. In addition, ATRT TM
provides a central repository for testing artifacts from test-case
development and documentation, to requirements traceability and
test-case results.
Accurately Analyze Massive Data Volumes
Complex, high-reliability systems often generate more test datathan can be analyzed using manual approaches. IDTs ATRT AM
automatically analyzes all test data using a model-based approach
that characterizes the systems behavior and evaluates the test
results against requirements and performance specifications.
Large complex systems can generate gigabytes or terabytes of
data that are not possible for people to evaluate. They have the
information needed, but as their systems grow, they discover they
cant evaluate everything because its too time-consuming and
labor intensive, said Gauf. Because ATRT AM is able to analyze
against requirements and performance specifications in an auto-
mated fashion, you can increase test coverage and reduce the time
it takes by an order of magni-
tude compared to manual
methods.
Testing and QA teams oftenonly have time to evaluate a
small subset of the data col-
lected, and as a result have the
potential to miss errors. ATRT AM is capable of analyzing all the test
data, pinpointing errors, and explaining how the test results differ
from the way the system is expected to perform.
Improve Test OutcomesAutomation speeds up testing, but by itself may not help organiza-
tions achieve critical-path goals. According to Gauf, technical
teams often start automating processes without considering what
theyre automating and why theyre automating them. Becausetheyre approaching automated testing incorrectly (e.g., not being
selective about the tests theyre automating, etc.), they mistakenly
conclude that automation is not making the type of impact the
organization expected.
IDT helps customers define an Automated Testing Strategy
thats tailored to their unique requirements and test programs.
With an Automated Testing Strategy in place, its possible to iden-
tify the critical-path tests that will yield the highest ROI based on
objectives.
You can automate anything, but you should be automating
what makes sense, said Gauf. If you want to release software
faster, well focus on those tests that are consistently in the critical
path for each software release, and require the most time and peo-ple. That way, you can dramatically reduce your testing cycle, and
you have the ability to repeat the tests in future releases.
IDT designs and builds automated tests in partnership with cus-
tomers internal teams. IDT also builds custom turnkey automat-
ed-testing solutions that customers can simply execute. The solu-
tions are built using ATRT technology, so they are capable of
supporting the entire testing life cycle of a system, including phas-
es such as component testing, system integration, system testing
and test result analysis, among others.
In addition to its products and services, IDT offers on-site and
remote training for customers who want to learn how to implement
ATRT effectively. The training programs range from introductory to
expert levels.
For more information about IDTs ATRT products and services,
visit www.idtus.com.
Increasing Testing Productivity with IDTsAutomated Test and Re-Test (ATRT)
THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK MARCH 15
If you want to release softwarefaster, well focus on those
tests that are consistently inthe critical path.
Bernie Gauf
TEST/QA AUTOMATIONGold Sponsor
-
8/12/2019 Software Quality Sourcebook
16/20
http://sdtimes.com/pdf/Ditch/1870 -
8/12/2019 Software Quality Sourcebook
17/20
Keeping pace with agile release cycles is challenging when test-
ing and test management cant be done in real time. Zephyrs
real-time test-management solutions provide instant visibility
into the actual status of tests 24x7, driving teams to improve work-
flows, processes and collaboration.
Testing teams used to be in one building or on one floor, but
now theyre spread out all over the world as a result of globally dis-
tributed teams and outsourcing, said Samir Shah, CEO of Zephyr.
When youre working in different time zones, traditional methods
of waiting for reports and facilitating status calls are inefficient.
Project teams need instant information to stay on the same page at
all times.All Zephyr productsZephyr Enterprise Edition, Zephyr for
JIRA and Zephyr Community Editionmeasurably improve pro-
ductivity and software quality.
Zephyr Enterprise Edition Improves Complex,Global Test ManagementZephyr Enterprise Edition manages all
aspects of software quality, and is suited for
departments and enterprises that run multi-
ple projects in parallel, have structured testing
cycles, and do automated testing or a combi-
nation of automated and manual testing. It
offers full two-way integration to JIRA.
Project teams still behave as if they work
in the same location, said Shah. They have
long conference calls that substitute for meetings in which they ask
for status reports; we automate these types of manual processes.
Using Zephyr Enterprise Edition, data-driven decisions can be
made while the information is still relevant. And, when anyone
anywhere makes a change to a requirement, test case, assignment,
defect or document, the data is automatically pushed out to dash-
boards so stakeholders can view the current status.
Updated information is automatically displayed on your dash-
board, so you dont have to press the refresh button or ask someone
for it; the data finds you, said Shah. Traditionally, testers have hadto send their daily reports to managers who package and send the
reports to project managers. PMs then extract pieces of data and
send it up to management. The whole process is fraught with laten-
cy and inefficiency.
With Zephyr Enterprise Edition, there are no reports to run or
metrics to calculate. It all happens automatically. Whats more, the
software includes a knowledgebase that stores apps, release ver-
sions, features, functions and tests so test assets can be reused. It
also offers built-in APIs for tight integrations to the users preferred
automation frameworks, providing manual and automated test
metrics in a single view.
People dont think about the knowledge that walks out the
door, said Shah. If you do not have a single repository that every-
one is accessing, you lose a lot of efficiency, especially if youre
working in a globally distributed fashion.
E-commerce platform provider GSI Commerce leveraged both
JIRA and Zephyr Enterprise Edition to manage testing across 30
projects, 90+ releases, and 12,000 test cases spread across a team of
280 people in three global locations. The company can now easily
access and reuse testing assets globally; create custom reports; and
customize the workflow in one system without losing the cus-
tomization in the other.
Zephyr Enterprise Edition is available both as a SaaS solution
and as an on-premise solution.
Zephyr for JIRA Provides Seamless Test ManagementZephyr for JIRA is a native add-on built exclusively for the JIRA 5platform. It completes end-to-end project management in JIRA by
adding testing to the overall planning, development, bug-tracking
and reporting processes. Zephyr for JIRA is best suited to project
teams that want to seamlessly
integrate testing into their
workflows.
Zephyr for JIRA gives you
sophisticated testing capabili-
ties inside JIRA, said Shah.
Project teams that have stan-
dardized on JIRA want to incorporate testing within the exact same
interface. We released it a year ago and, since the day it waslaunched, it continues to remain as either the No. 1 or No. 2 Top
Grossing add-on each week on the Atlassian Marketplace.
Transportation and language service company ProCare used
Zephyr for JIRA to add more structure to its testing process and
to create a repeatable process. Before using Zephyr for JIRA, Pro-
Care had been using JIRA, Confluence and spreadsheets. Because
a lot of the work was manual, bugs would surface after releases.
Now, instead of guessing when test cycles might be completed,
ProCare can instantly view the top defects affecting tests, and use
burndown charts to predict when test cycles will be completed
right within the JIRA interface. The company is benefitting from
more detailed analyses of bugs, easy navigation between test cas-es, and the traceability that comes from linking test cases to
issues.
Get Zephyr Community Edition FreeInstead of foregoing test-management tools because the out-of-
pocket costs are too high, thousands of users have turned to Zephyr
Community Edition, which is free for project teams of up to 10
users who want the benefits of real-time updates and the ability to
build a knowledgebase. The community edition can be easily con-
verted to Zephyr Enterprise Edition with a new license key.
We decided to offer a free version for small teams that needed the
functionality, flexibility and maintenance of an enterprise-class solu-
tion but were only working with an open-source budget, said Shah.Learn more at www.getzephyr.com.
Real-Time Test Management from
Zephyr Drives Instant Visibility
THE 2013 SD TIMES SOFTWARE QUALITY SOURCEBOOK MARCH 17
If you do not have a singlerepository that everyone isaccessing, you lose a lot ofefficiency.
Samir Shah
TEST MANAGEMENTGold Sponsor
-
8/12/2019 Software Quality Sourcebook
18/20
@SPTechConA BZ Media Event www.sptechcon.com
August 11-14, 2013Sheraton Boston
Choose from more than 85 classes
and tutorials that will let youput together your own customSharePoint training experience.
Whether you need help migrating to SharePoint 2013
or are still making the most out of SharePoint 2010,
you will find the SharePoint training you need at
SPTechCon. Many of our speakers are Microsoft MVPs,
and all are SharePoint experts. A rigorous review
of the classes after each conference allows us to weed
out the weaker classes and speakers.
Classes at SPTechCon average almost 9 on a 10-point
scale, and classes scoring below an 8 are not invited
back. No wonder SPTechCon is the largest independent
SharePoint training conference in the world.
The BestSharePoint Training
in the World!
SPTechCon is a trademark of BZ Media LLC. SharePoint is a registered trademark of Microsoft.
At SPTechCon, all the experts get together in one place and
present fantastic, current and relevant content.Keith Budurka, IT Director, Infrastructure, Benco Dental
If your goal is to learn something from a trusted leader in
the industry and get an opportunity to network, you must go
to SPTechCon.Brian Perryman, SharePoint Developer, Health South Corporation
SPTechCon is a great event and well worth the admission fee.Chris Kauke, System Consultant, MassMutual
http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/http://www.sptechcon.com/ -
8/12/2019 Software Quality Sourcebook
19/20
T
odays businesses run on software, and yet errors still cause
service disruptions, cost overruns and customer-support
headaches. With Orasis help, enterprises can ensure theirapplications run predictably at scale.
Since 2002, Orasi has helped enterprise customers create soft-
ware-testing environments that reduce the cost and risk of software
failures. The company resells HPs Application Lifecycle Manage-
ment product line and provides support for ALM solutions as an HP
Support Partner. Orasi is an HP Specialist Enhanced Partner, and
was named HP Support Partner of the Year in 2009, 2011 and 2012,
and Partner of the Year (US) in 2011. Orasi also maintains a dedicat-
ed SAP practice aimed at enterprises that are deploying or upgrad-
ing SAP solutions, and has achieved SAP Gold Level Partner status.
Orasis testing-service capabilities span the entire application
life cycle, so clients can get expert assistance from initial imple-
mentations to enterprise rollouts. To help customers get the most
from their investments, Orasi developed its Rapid Return on Invest-
ment (R2OI) approach that translates customers business objec-
tives into a flexible set of services that enable faster time to market,
reduced risks and lower costs.
Organizations realize they have to make software quality a pri-
ority because software is the lifeblood of their business, said Nick
Kavadellas, president and CEO of Orasi. As a leading software
reseller and professional services company, we have deep knowl-
edge across functional, performance, mobile device, security, and
specific application testing.
Orasi Solutions Mirror Enterprise RequirementsCustomer-requested Orasi testing solutions suit the modern
demands of enterprise teams. Its JIRA Bridge for HP Quality Centersynchronizes defect information between Atlassians JIRA and HP
Quality Center to improve communication between QA and devel-
opment teams. The tool creates issues when test cases fail, and pro-
vides metrics that help improve the reliability of release cycles.
Software teams want to link test cases back to development
artifacts, said Kavadellas. Because JIRA has been so popular, cus-
tomers asked us to integrate JIRA and HP Quality Center. Now,
more than 100 customers are using it.
Orasi also offers a cloud-based version of HP LoadRunner called
Orasi CloudPerform, which can be purchased in 24-hour incre-
ments. The tool allows customers to capture, run and analyze bottle-
necks in cloud-based applications. It supports Silverlight and Flex;
Packages are also available for Citrix and Microsoft RDP testing.
Take Advantage of Affordable Onshore TestingOrasi established a rural testing center in Greensboro, N.C., five
years ago. The test center now employs more than 120 engineers
who assist clients throughout the U.S.
Software development organizations are realizing how impor-
tant it is to have an effective methodology and process in place,
said Kavadellas. Four years ago, mobile apps were considered fun.
Now, you have to make sure theyre mission-critical.
Learn more at www.orasi.com.
MOBILE TEST/QA
Master Software Quality with Orasi
Silver Sponsor
http://sdtimes.com/pdf/CloudCurrent/1871 -
8/12/2019 Software Quality Sourcebook
20/20
Get the best real-world Androiddeveloper training anywhere!
Choose from more than 75 classesand tutorials
Network with speakers and other
Android developers
Check out more than
40 exhibiting companies
BOSTON May 28-31,2013The Westin Boston Waterfront
AnDevCon is a trademark of BZ Media LLC Android is a trademark of Google Inc Googles Android Robot is used under terms of the Creative Commons 3 0 Attribution LicenseRegister NOW at www.AnDevCon.com
AnDevCon is one of the best
networking and information hubs
available to Android developers.Nate Vogt, Android Developer,
Willow Tree Apps
KEYNOTE!
http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/http://www.andevcon.com/