Post on 06-Jul-2018
8/17/2019 Market Guide for Mobile Appl 273008
1/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
G00273008
Market Guide for Mobile Application Testing
ServicesPublished: 29 October 2015
Analyst(s): Susanne Matson, Maritess Sobejana
Improving customer experience in a digital economy is critical to success,
driving organizations to invest in mobile application development. Sourcing
managers must understand the complexity of the mobile testing
requirements to ensure service providers can meet business expectations.
Key Findings■ The rapid developments of mobile and other digital technologies is necessitating agile and
DevOps methodologies, requiring organizations to reskill and maintain the relevant skills in-
house. However, lack of available skills in the market is driving higher usage of external service
providers since there are multiple skills needed for mobile development and testing — plus the
requirements constantly change.
■ Mobile testing services are inherently different to traditional application testing services, as they
require added complexity, such as the testing of multiplatforms, networks, OS, as well asdifferent devices. User experience is also key in mobility as it may generate unique requirements
for the testing process as well as considerations for testing by geography and security testing.
■ Lack of forward planning and engaging sourcing as well as IT, proactively drives the business to
bypass quality test planning to meet time-to-market needs. This results in higher risk with a lack
of testing metrics in the contract in order to hold service providers accountable.
Recommendations
Sourcing managers should:■ Include experience in agile and iterative development methods as key selection criteria when
choosing mobile application testing service providers, since it requires a different skill level of
resources than traditional testing services. Due to high collaboration, a nearshore or on-site
presence is necessary to be fully effective.
■ Document the business requirements — understanding the bimodal aspect of the development
and testing required. Since mobile application testing is part of both Mode 1 and Mode 2,
8/17/2019 Market Guide for Mobile Appl 273008
2/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
addressing both the need for stability as well as agility. This may impact the selection criteria as
well as the provider experience desired.
■ Engage with the business early in order to develop an efcient testing strategy. Avoid risk of
business bypassing in accessing these applications without sufcient planning for testing
quality — in lieu of speed.
Market DenitionGartner's 2015 CIO survey revealed that currently, mobile ranked as the fth highest priority for CIO
spend. Naturally, ensuing testing demands must follow. However, there is lack of awareness by
clients on the complexity of testing required in mobile development, as well as how it differs from
traditional application testing, so it makes it difcult to select the right service provider.1
Mobile testing services are different, often requiring more complex than traditional application
testing services since they require new skill sets and considerations — such as the testing ofmultiplatforms, networks, OS as well as different devices. These services are needed to test the
function, performance, compatibility and so forth, of a mobile application — inclusive of both
manual and automated tests. Therefore, internal testing organizations need to consider mobile
application testing service providers as part of their quality strategy.
User experience is key in mobility by creating unique requirements for the testing process. With
digital business and customer experience, applications and data is increasingly available and visible
to external constituents — especially customers. Thus, quality, reliability, security and adaptability
needed for external use of systems is far beyond internal usage — and this is especially true with
mobile applications. The competitive landscape of mobile application testing service providers
comprises a variety of traditional and nontraditional players including:
■ Full service IT service providers
■ Pure-play testing or mobile testing service providers
■ Digital that offers mobile testing
■ Testing specialists who specialize in mobile testing entirely in the cloud
■ Crowdtesting providers
Buyers of mobility testing services range from business leaders, sourcing managers, application
development managers and testing managers. Services to support clients' mobile application,quality control and quality assurance to ensure the high quality demanded by consumers, clients,
stakeholders and end users are highly requested by buyers today. This research includes all test
consultancy, advisory, implementation, execution and management services related to mobile apps,
we also examine how it functions on various devices.
For the purpose of this Market Guide, we do not include crowdtesting service providers. However,
we recognize that this can be effectively used for mobile testing and these providers are covered in
Page 2 of 33 Gartner, Inc. | G00273008
8/17/2019 Market Guide for Mobile Appl 273008
3/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
"Market Guide for Crowdsourced Application Testing Services." Additionally, testing product
companies and providers are covered in "Market Guide for Mobile Application Testing."
Market Direction Application development and business managers (sales, marketing operations that include mobile
applications) are facing pressure to ensure the consistent high quality and user experience of mobile
applications. Mobile is one of the key technologies encompassed within digital business initiatives;
where demands of the business include shorter time to market, keeping costs low in often complex
testing environments along with maintaining the high levels of quality, are top of mind for business
leaders. Financial losses, legal issues and irreparable brand image would be a high price to pay if
mobile application fails to deliver the outcomes expected.
Since mobile applications drive digital business and success is determined by user experience, they
are generally led and adopted by business units rather than the IT department. IT and sourcing
therefore, needs to be fully engaged and be knowledgeable about the requirements of different
business units and their needs in order to deliver high-quality mobile apps. Thus, this entails
working closely with the business units on mobile testing services demands — which is
continuously challenging traditional IT management strategies — as it requires agile and exibleorganizations.
This market is developing rapidly. Currently, it is still fairly fragmented and unstructured, which
means more potential confusion for buyers. The need to support these applications is quite new so
the capabilities to manage the required testing are not often present within the organizations
internally. Anticipating further market maturity, there will be many tactical relationships forming to
meet intense demand.
Mobile Application Testing Tools Market Will Drive Use of External Testing Services
The mobile tools market is expected to rapidly grow with new entrants continuing to appear with
varying product offerings that will inuence the services market over the subsequent ve-year
period (see "Market Guide for Mobile Application Testing"). The reality is that before the market even
reaches maturity, the initial confusion and strong ramp-up in demand is mixed with the need for
essentially new skills and capabilities. Looking even further into the future, mobile technologies (and
how they are integrated) are likely to continue evolving rapidly, and will be inuenced by other
technologies such as the Internet of Things (IoT), beacons, wearables, smartwatches and machine
learning. These aspects will drive increased reliance on service providers. Typically, this is the case
with any emerging market.
Lack of Digital Skills Is a Major Incentive to Consider Mobile Application TestingServices
The investments IT service providers make into digital technologies and in continuously training
their staff is a key driver for buying mobile application services. Since digital skills such as mobile
application development and testing require updated products and apps, it needs ongoing
Gartner, Inc. | G00273008 Page 3 of 33
8/17/2019 Market Guide for Mobile Appl 273008
4/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
investment to keep talent and technology refreshed frequently — simply due to the sheer speed of
innovation.
Test automation skills are becoming more essential alongside digital business. The speed with
which development of digital technologies occur exposes a new reality for organizations, whereby
they will not be able to reskill or retrain fast enough — there are simply too many skills required thatgo through constant changes. Speed is also accelerating, so the time to market for a mobile project
can be a matter of days or weeks, as opposed to legacy applications in the traditional sense.
Organizations need to make a choice about whether they are willing or able to maintain all these
skills in-house. The market broadly suffers from a shortage of various specialized mobile application
testing skills, which proves a signicant challenge — even for service providers. However, because
this is their business, they make focused investments in these areas placing them in a better
position to train, maintain and acquire the right skills in comparison with internal testing
organizations.
Bimodal IT Will Drive the Need for AgilityMobile technologies will evolve rapidly and principles such as bimodal IT will help to facilitate
enterprise agility. These constant changes, both at the technology and consumer level, places
signicant demands on organizations, which make dening a mobile strategy difcult. Gartner has
covered the area of bimodal in research noting that there are two modes to consider — Mode 1 and
Mode 2 — which, when combined, serve to address both the stability and the agility requirements
within organizations.
As organizations develop bimodal IT capabilities, they will need to form application development
teams with different objectives, processes, metrics, disciplines, and consequently, varying needs in
order to support both Mode 1 and Mode 2 projects. In mobile application development, Mode 1
refers to a stable infrastructure (API, mobile app integration layer) where mobile apps can beintegrated with back-end systems; Mode 2 refers to the front-end client of the app (app UI and user
experience [UX]) that the user interacts with.
Mode 2 includes more emerging areas of interest to the organization in terms of new business
development with quicker turnaround times, agile approaches, new service methods and delivering
front-end applications. Therefore, a different approach to the various providers needed in the
sourcing mix is likely to be necessary.
We often nd providers moving into offering specialized services or modes of delivery, as well as
more exibility and ease of use. Application development and sourcing managers need to be aware
of the new opportunities and providers in this emerging market, as well as know how best to workwith these providers and also obtain services that cater for the varying needs of the organization.
Mobility itself is also constantly changing, at both the technology level and the societal level, with
the resulting demands on organizations. This uid environment makes dening a mobile strategy
complex. How can the conicting needs of enterprise stability and rapid response be resolved? The
use of a bimodal strategy — one mode addressing stability and the other agility — allows
organizations to maximize development efciency and respect information control policies, yet
addressing the urgency of volatile, competitive, seasonal and unplanned requirements.
Page 4 of 33 Gartner, Inc. | G00273008
8/17/2019 Market Guide for Mobile Appl 273008
5/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Adoption of Agile Methodologies Require More Automation
Developing bimodal IT capabilities demand that IT organizations begin to deal with agile methods
for application development and delivery. Agile methodologies include frequent meetings between
the development team and application team, with much shorter cycles for updates and revisions.
This results in the need for increased levels of automation and tooling support, to achieve therequired speed of delivery. However, automation requires different tools and frameworks as well as a
different skill set to deliver manual execution. Moreover, initial cost for automation is high —
licensing cost, training, creation of scripts, maintenance, and so on. The automation tools
developed for traditional testing services might not work for mobile application testing, so providers
are building new or extending the tools they already have, to increase the level of automation —
leading to accelerated testing, increased efciency and reliability of testing. Automation tools are
frequently requested by clients, so creating a portfolio of strong offerings can prove to be the
differentiator for providers. Resources provided by the service provider must also be able to work in
an agile, dynamic and uid development environment. This is a completely different way of working
with service providers. Shorter sprints and shorter cycles, as well as a much more collaborative
working methodology mean different requirements of service providers. With an agile approach, it isnot best practice to use service providers for all of the work — it helps to have internal quality
sponsors that will oversee the projects. Effective engagement is dependent upon collaboration
between business leaders, application developers and testers. Working together, especially on the
same site, often allows for face-to-face collaboration that reduces misinterpretations of
requirements and enables testing during development. Having project managers on the team who
can lead and interpret results is essential.
Market Analysis
With the expansion of new technologies such as mobile being further integrated into organizationsthat are forming digital business opportunities, there are multiple challenges with organizations
needing to ensure that their mobile applications are of good quality.
Testing Mobile Applications Is Far More Complex Than Traditional ApplicationTesting
The inherent complexities in testing mobile applications are compounding the challenges to
maintain quality assurance. When testing mobile applications, the process must cover multiple
platforms, multiple networks of varying stability, different OS including frequent OS upgrades,
including the different devices the apps are running on (devices like Samsung Electronics, Nokia,
Apple and HTC). All these options and combinations require a different view. A mobile application ispart of a system, for example, mobile payment that may involve the application of a Near Field
Communication (NFC) reader and a point of sale (POS) system. Furthermore, a prominent testing
requirement in the mobile applications space is user experience. Mobile is increasingly a way for
organizations to communicate with clients, win business in new areas or reach various stakeholders
— so user experience is paramount. Therefore, usability and exibility of mobile applications is top
of the agenda. In the future, the user experience will likely become even more central, with the
devices and services less so. Further considerations such as security (like testing the robustness of
Gartner, Inc. | G00273008 Page 5 of 33
8/17/2019 Market Guide for Mobile Appl 273008
6/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
data protection solutions) are potentially even more important for mobile applications, since user-
facing components of systems will move increasingly outside of the rewall. In addition,
geographical and language aspects are elements that also require contemplation in the testing of
mobile applications.
To effectively test mobile applications, clients are using various methods, such as creatingcompetency centers for critical platforms (iOS, Android), carriers (Verizon, BT), as well as security
and digital design. Clients also use mobile testing service providers to complement internal
capabilities and/or crowdsource critical missing competencies.
Fundamental Changes in Engagement Models and Pricing
Due to the nature of mobile applications — in terms of shorter time to market, higher speed, smaller
projects and more collaborative, agile work — it can be more effective to use a combination of
services, techniques and providers. The RFI and RFP processes for traditional application testing
projects are often too long, therefore quick decisions frequently need to be made. Also, exibility in
contracting is likely more important when negotiating with mobile testing pure-play rms or digitalagencies. For guidance on some of the key criteria that can be used when selecting a mobile
application testing service provider, see "How to Choose a Mobile Application Testing Service
Provider."
An increasing number of smaller, more nimble providers are moving into this space and can often be
very effective as part of a sourcing strategy. These providers come from varying backgrounds and
can be good, exible, quick, cost-effective and skillful additions. However, note the variety of
provider and carefully consider which type of provider would make the most suitable t and what
they specialize in.
The use of on-site or onshore resources as part of the sourcing mix is also essential to ensuresuccess. Agile and DevOps methodologies (with the speed generally needed in these projects),
have a positive effect on service providers by winning deals that have sufcient resources closer to
the client. As service providers and clients become more experienced by working on agile projects,
offshore usage may be increased over time to control costs. As a result, we see service providers
creating more delivery centers closer to the client (if needed) and testing professionals on-site,
embedded within project teams with utilization of offshore resources.
Cloud is a model frequently considered for mobile application testing. Here devices can be run on
multiple systems in the cloud and tested by creating a Web-based mobile environment on a
simulator to access the mobile app. Public cloud is of interest to many clients that do not mind the
potential lack of control or privacy issues, while data-privacy sensitive clients will prefer the privatecloud. The cloud can be easily accessed; it is cost-efcient and scalable, fast and effective.
The exibility in pricing and scalability that testing-as-a-service (TaaS) represents is frequently used
for mobile application testing and as long as it is clear what the client is paying for, this can prove to
be a good alternative.
Page 6 of 33 Gartner, Inc. | G00273008
8/17/2019 Market Guide for Mobile Appl 273008
7/33
8/17/2019 Market Guide for Mobile Appl 273008
8/33
8/17/2019 Market Guide for Mobile Appl 273008
9/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
■ Media
■ Professional services
■ Retail
■ Sports and entertainment
■ Telecommunications
b-acceptance
( www.b-accept-ance.com )
Pure-play test-ing serviceprovider
France Focuses on allsegments
■ Consumer products
■ Healthcare
■ High tech
■ Media
■ Professional services
■ Retail and wholesale distribution
■ Ranorex
■ TestLink
■ HP Quality Center
Capgemini
( www.capgemini.com )
Global full-service provid-er
France Primary focus is2,500 employeesand above
■ Banking
■ Insurance
■ Communications
■ Other nancial services
■ Public sector
■ HP
■ Perfecto Mobile
■ DeviceAnywhere
■ Appium
■ Mobile Labs
CGI
( www.cgi.com )
Global full-service provid-er
Canada Primary focus is2,500 employeesand above
■ Financial services
■ Communications
■ Public sector
■ HP
■ Perfecto Mobile
■ Keynote
Gartner, Inc. | G00273008 Page 9 of
http://www.cgi.com/enhttps://www.capgemini.com/http://www.b-acceptance.com/http://www.b-acceptance.com/
8/17/2019 Market Guide for Mobile Appl 273008
10/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
■ Healthcare
■ Energy
Cigniti Technologies
( www.cigniti.com )
Pure-play test-
ing serviceprovider
India Primary focus is
organizations be-tween 1,000 to2,500 employees
■ Banking
■ High tech
■ Insurance
■ Professional services
■ Retail
■ Travel and transportation
■ Experitest (SeeTest)
■ Ranorex
■ TestPlant
■ Neotys
■ Perfecto Mobile
■ Mobile Labs
■ HP
■ Soasta
Cognizant
( www.cognizant.com )
Global full-service provid-er
U.S. Primary focus is2,500 employeesand above
■ Automotive
■ Banking
■ Consumer products
■ Healthcare
■ Higher education
■ High tech
■ Insurance
■ Life science
■ Media
■ Experitest
■ Mobile Labs
■ Perfecto Mobile
■ Applause
Page 10 of 33 Gartner, Inc. | G00273
http://www.cognizant.com/http://www.cigniti.com/
8/17/2019 Market Guide for Mobile Appl 273008
11/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
■ Retail
■ Sports and entertainment
■ Telecommunications
■ Travel and transportation
■ Utilities
CresTech Software
Systems
( www.crestechglo-bal.com )
Pure-play test-ing serviceprovider
India NA ■ Banking
■ Insurance
■ Media and entertainment
■ Telecommunications
■ Travel
■ Amazon Web Serv-
ices
■ Armorize Technolo-
gies
■ IBM
■ Orane Consulting
CSC
( www.csc.com )
Global full-service provid-er
U.S. Primary focus is1,000 employeesand above
■ Healthcare
■ Insurance
■ Banking
■ High tech
■ Public services
■ Manufacturing
■ Transportation
■ Retail
■ Perfecto Mobile
■ Soasta
Gartner, Inc. | G00273008 Page 11 of
http://www.csc.com/http://crestechglobal.com/http://crestechglobal.com/
8/17/2019 Market Guide for Mobile Appl 273008
12/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
Deloitte
( www.deloitte.com )
Global full-service provid-er
U.S. Primary focus is2,500 employeesand above
■ Financial services
■ Public
■ Healthcare
■ Manufacturing
■ Communications
■ Mobile Labs
■ Perfecto Mobile
Epam
( www.epam.com )
Regional midt-ier full or sub-set IT serviceprovider
U.S. Primary focus is2,500 employeesand above
■ Banking
■ Consumer products
■ Healthcare
■ High tech
■ Life sciences
■ Media
■ Retail
■ Telecommunications
■ Travel and transportation
■ Experitest
■ Xamarin
■ Perfecto Mobile
■ Selenium (open
source)
■ Appium (open source)
HP
( www.hp.com )
Global full-service provid-er
U.S. Focuses acrossseveral ranges
■ Healthcare
■ Public sector
■ Communications
■ Media and entertainment
Perfecto Mobile
Page 12 of 33 Gartner, Inc. | G00273
http://www8.hp.com/us/en/home.htmlhttp://www.epam.com/http://www2.deloitte.com/global/en.html?icid=site_selector_global
8/17/2019 Market Guide for Mobile Appl 273008
13/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
■ Transportation
IBM
( www.ibm.com )
Global full-service provid-er
U.S. Primary focus isorganizations withmore than 10,000
employees
■ Banking
■ Consumer products
■ Healthcare
■ Insurance
■ Media
■ Retail
■ Sports and entertainment
■ Telecommunications
■ Travel and transportation
■ Perfecto Mobile
■ IBM Rational
■ DeviceAnywhere
InfoStretch
( www.infostretch.com )
Mobile serviceprovider
U.S. NA NA ■ Kony
■ Perfecto Mobile
■ Sauce Labs
■ Soasta
■ Mobile Labs
■ PhoneGap
■ Atos
■ Veracode
Gartner, Inc. | G00273008 Page 13 of
http://www.infostretch.com/http://www.ibm.com/
8/17/2019 Market Guide for Mobile Appl 273008
14/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
■ Xamarin
Infosys
( www.infosys.com )
Global full-service provid-er
India Primary focus is2,500 employeesand above
■ Automotive
■ Banking
■ Consumer products
■ Healthcare
■ High tech
■ Industrial machinery
■ Insurance
■ Life sciences
■ Oil and gas
■ Professional services
■ Public sector
■ Retail
■ Sports and entertainment
■ Telecommunications
■ Utilities
■ Experitest
■ Mobile Labs
■ HP
■ Perfecto Mobile
■ Jamo Solutions
■ Sauce Labs
Kualitatem
( www.kualitatem.com )
Pure-play test-ing serviceprovider
U.S. Primary focus isless than 500 em-ployees
■ Banking
■ Healthcare
■ HP
■ IBM
■ Selenium
Page 14 of 33 Gartner, Inc. | G00273
http://www.kualitatem.com/http://www.infosys.com/
8/17/2019 Market Guide for Mobile Appl 273008
15/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
■ Media
NTT Data
( www.nttdata.com )
Global full-service provid-er
Japan Evenly spreadfrom organizationswith 500 employ-
ees, but most or-ganizations havemore than 10,000employees
■ Healthcare
■ Financial services
■ Insurance
■ Telecommunications
■ Perfecto Mobile
■ DeviceAnywhere
■ Jamo Solutions
Olenick & Associates
( www.olenick.com )
Regional midt-ier full or sub-set IT serviceprovider
U.S. NA ■ Banking
■ Capital markets
■ Healthcare
■ Utilities
■ HP
■ Microsoft
■ IBM
■ Soasta
■ Perfecto Mobile
■ Tahoe Partners
QASource
( www.qasource.com )
Pure-play test-
ing serviceprovider
U.S. Primary focus is
up to 2,500 em-ployees
■ Banking
■ Consumer products
■ Engineering
■ Construction and operations
■ Healthcare
■ Higher education
■ High tech
■ Insurance
QASource is tool-agnostic, al-
though it works with multiple part-ners
Gartner, Inc. | G00273008 Page 15 of
https://www.qasource.com/http://www.olenick.com/http://www.nttdata.com/
8/17/2019 Market Guide for Mobile Appl 273008
16/33
8/17/2019 Market Guide for Mobile Appl 273008
17/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
■ Manufacturing ■ Keynote Mobile Test-
ing
RapidValue Solutions
( www.rapidvaluesolu-
tions.com )
Regional midt-ier full or sub-
set IT serviceprovider
U.S. Primary focus isorganizations with
either less than500 or between1,000 and 2,500employees
■ Banking
■
Consumer products
■ High tech
■ Insurance
■ Media
■ Oil and gas
■ Professional services
■ Retail
■ Telecommunications
■ Appium
■
Perfecto Mobile
■ SeeTest
■ BlazeMeter
■ SoapUI
SLK Software
( www.slk-group.com )
Regional midt-ier full or sub-
set IT serviceprovider
India Primary focus isorganizations with
more than 1,000employees
■ Automotive
■
Banking■ Industrial machinery
■ Insurance
■ Retail
■ Utilities
■ Wholesale distribution
■ Perfecto Mobile
■
Appium■ HP UFT
Gartner, Inc. | G00273008 Page 17 of
http://www.slk-group.com/http://www.rapidvaluesolutions.com/http://www.rapidvaluesolutions.com/
8/17/2019 Market Guide for Mobile Appl 273008
18/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
StarDust
( www.stardust-test-ing.com )
Pure-play test-ing serviceprovider
France Primary focus isorganizations withmore than 2,500employees
■ Banking
■ Healthcare
■ Insurance
■ Life sciences
■ Retail
■ Travel and transportation
■ Wholesale distribution
NA
SQS
( www.sqs.com )
Pure-play test-ing serviceprovider
Germa-ny
Primary focus is2,500 employeesand above
■ Banking
■ Other nancial services
■ Insurance
■ Automotive
■ High tech
■ Services
■ Other manufacturing
Perfecto Mobile
Syntel
( www.syntelinc.com )
Regional midt-ier full or sub-set IT serviceprovider
U.S. Primary focus isorganizations withmore than 10,000employees
Banking ■ Kony
■ Microsoft
■ Salesforce
■ Experitest
Page 18 of 33 Gartner, Inc. | G00273
http://www.syntelinc.com/http://www.sqs.com/en-group/index.phphttp://www.stardust-testing.com/http://www.stardust-testing.com/
8/17/2019 Market Guide for Mobile Appl 273008
19/33
8/17/2019 Market Guide for Mobile Appl 273008
20/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Provider/Website Category Head-
quar-
ters
Client Segments Industries (Primary Focus as Indicated by
Provider)
Partnerships
Virtusa
( www.virtusa.com )
Regional midt-ier full or sub-set IT serviceprovider
U.S. Primary focus isorganizations withover 10,000 em-ployees
■ Automotive
■ Banking
■ Healthcare
■ Insurance
■ Media
■ Telecommunications
■ Experitest
■ HP Mobile Center
■ Appium
Wipro
( www.wipro.com )
Global full-service provid-er
India Primary focus isorganizations withmore than 10,000employees
■ Retail
■ Banking
■ Insurance
■ Healthcare
■ Mobile Labs
■ Perfecto Mobile
ZenQ
( www.zenq.com )
Pure-play test-ing serviceprovider
India Primary focus isorganizations withless than 500 em-
ployees
■ Banking
■ Higher education
■ Healthcare
■ Insurance
■ Retail
■ Appium
■ Silk Mobile
■ pCloudy
■ Keynote
NA = not applicable
Source: Gartner (October 2015)
Page 20 of 33 Gartner, Inc. | G00273
http://zenq.com/http://www.wipro.com/http://www.virtusa.com/
8/17/2019 Market Guide for Mobile Appl 273008
21/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Global Full-Service Providers
Full service providers offer various IT services including testing services practice with the mobile
application testing capabilities as a subset of that practice. Mobile testing is an area of strong
demand in the market and most of the full-service providers of a certain size offer mobile testing
services. (See Table 1 for further examples of providers.) Buyers can consider using these providersfor larger, complex or global deals.
Atos
www.atos.com
Atos is a global service provider that has a testing practice with over 8,000 resources. Atos also has
a focused mobile application testing practice with just under 800 resources. Atos has most of its
regional capability based in Europe (U.K., Belgium, the Netherlands and Luxembourg [Benelux],
Spain, France), although an increasing part of its revenue stems from North America and various
other countries. More strategic focus has been placed on the Atos testing practice over the pastcouple of years, with increased investment into the practice, its rebranding into an independent unit
and a raised prole within the organization. Mobile testing is one of the areas of innovation that the
provider stated it wanted to focus on from 2015 onwards, as it leverages its global strategic
partnership with InfoStretch (a mobile application service provider that is detailed in the Mobile
Service Providers section, it delivers mobile application testing services). Atos has a mobile testing
center of excellence (COE) in India and specialized smart mobility delivery centers. Atos mobile
testing services include end-to-end automated mobile testing solutions, as well as context-driven
mobility services.
Cognizantwww.cognizant.com
Cognizant is a global service provider with a large testing practice, which includes mobile
application testing services. Cognizant has an India-centric delivery model, with most of its revenue
originating in its home market of North America but also offers services in multiple other regions and
countries.
Cognizant continues to move further into the areas of digital and mobile by leveraging its
technology labs, assets, intellectual property (IP) and partner ecosystem. Currently, the provider has
10 global labs focusing on mobile activities located in the U.S., China, India, U.K. and a few other
countries in Europe and Asia/Pacic. Cognizant offers an end-to-end mobile testing solutionleveraging internal developed IP, including automation for mobile testing via its "mint" platform and
"BlueSim," a solution for connected device virtualization using Bluetooth technology.
Examples of testing services offered include functional testing, automation testing, eld network
testing, user experience testing, nonfunctional testing, security, cross browser/platform capability
testing and application validation.
Gartner, Inc. | G00273008 Page 21 of 33
http://www.cognizant.com/http://www.cognizant.com/http://www.atos.com/
8/17/2019 Market Guide for Mobile Appl 273008
22/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
IBM
www.ibm.com
IBM is a full service global provider with various service areas, including testing and mobile
application testing services. IBM is a global provider that offers local support as well as globaldelivery centers. Mobile application testing is one of the current strategic focus areas of IBM. It uses
its portfolio of applications to support clients — from consulting and engagement through to IBM's
mobile test factory. IBM has developed mobile test facilities across India, Europe, U.S. and Asia/
Pacic, where it performs testing on multiple browsers, devices and platforms, including offerings
such as functional, performance and network testing services. IBM leverages its Combinatorial Test
Design (CTD) solution for reducing mobile test cases in the mobile test centers and it has also
worked with one of its partners, eggPlant, to develop a mobile automation solution.
An example of a recent initiative within the portfolio of mobile testing services includes its Apple
Garage, in partnership with Apple. The Apple Garage, which includes testing automation, integrates
IBM and Apple tools for continuous testing built on agile practices.
Infosys
www.infosys.com
Infosys is a full service global provider with a testing practice, as well as mobile testing capabilities.
Infosys includes just under 700 resources in its testing practice that are dedicated to mobile
application testing services. Infosys has extensive offshore centers which it leverages for clients
primarily based in North America, Australia and Europe — although it can also offer on-site or
nearshore resources where required. Within the service offerings of digital assurance, it offers
mobility quality assurance (QA) services, including mobile device testing, mobile app QA and mobile
QA consulting. These services comprise offerings such as functionality, platform integration,
performance, localization, interoperability, usability, security, UX benchmarking, network and
automation testing services or provide support around creating a mobile test strategy. Within its
mobile testing practice, Infosys has invested to accelerate and improve its effectiveness on client
engagements by creating various tools and frameworks. These include a mobile QA defect
repository tools, Mobile Scenario Testing Rapid Prototyping Engine (Mobile STRiPE) for early
validation; mobile application performance monitoring (mAPM) tool for device side performance
monitoring; network impact testing solution (iNITS) for testing mobile applications under varying
network conditions; a mobile platform specic validation framework as well as "iMobiLoad," a
performance QA tool and "CARES" a user experience test framework. Infosys also includes the
offering, Test Unit (TU)-based charging model, and another offering, integrated mobility-testing lab
on cloud, which can be accessed from customer locations worldwide.
Tata Consultancy Services
www.tcs.com
Tata Consultancy Services (TCS) is a global full service provider. It has one of the largest testing
practices in the market and its portfolio also includes mobile application testing services with
approximately 1,600 resources. TCS has recently scaled up and reinvented its digital assurance
Page 22 of 33 Gartner, Inc. | G00273008
http://www.tcs.com/Pages/default.aspxhttp://www.infosys.com/http://www.tcs.com/Pages/default.aspxhttp://www.infosys.com/http://www.ibm.com/
8/17/2019 Market Guide for Mobile Appl 273008
23/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
offerings — including mobile — by introducing specialization skills into its practice for mobile
testing, investing in a customer collaboration center at Santa Clara and a customer experience
center at Chennai Lab, and also building partnerships.
TCS incubates startups and partnerships in the mobile ecosystem and uses its dedicated mobile
testing labs in India and U.S. to train its workforce, support delivery, handle proof of concept (POC)programs and actually show customers how they test and assure in this digital space. Services are
delivered in traditional models as well as mobile-testing-as-a-service.
TCS has created mobility frameworks, other tools and IP to support delivery in this practice. Mobile
Reusable Automation Framework for Testing (mRAFT) is a framework for mobile test automation.
"iBrowse" provides automated cross-browser compatibility testing and result verication across
mobile browsers, it groups test cases based on business functions while "TCS Hy5 Test," which is
part of the TCS Hy5 Suite, enables the development, testing and deployment of applications on a
variety of mobile platforms through automation.
TCS offers assurance services on various mobile platforms, devices and applications, which includeservices such as network/environment virtualization, automation, performance, security,
accessibility, functional and nonfunctional testing on and off the cloud.
TCS primary focus is end-to-end mobility assurance engagements. Typically, TCS's engagements
span all aspects of assuring the mobile business, although it only addresses certain parts of the
mobile application testing life cycle in certain engagements.
Tech Mahindra
www.techmahindra.com
Tech Mahindra is a full service provider with a mobile application testing practice.
This company uses both on-site delivery along with its mainly Indian global delivery centers to
support its clients. Most of Tech Mahindra's revenue originates in North America but it also has a
large client base in some countries in EMEA. Its mobile application testing practice in Europe is, in
fact, its largest in terms of revenue. Mobile application testing services operate in a center of
excellence with 60 dedicated resources. The company has developed an automation framework,
"Mobile Application Testing Ecosystem" (MATE) that focuses on cloud-based automation-testing
services. There is a dedicated lab available for testing with an inventory of mobile devices. The
practice has capabilities for functional, automation, localization, user experience, compliance,
performance, security and cloud enabled testing.
Regional Midtier, Full or Subset IT Service Providers
These are smaller service providers with testing service capabilities including mobile. They offer
services in one region, one country and/or with a more specialized services portfolio. This group of
providers can be a good option both for small and midsize organizations looking for a local deal, but
some of these providers are also large enough to manage those bigger deals.
Gartner, Inc. | G00273008 Page 23 of 33
http://www.techmahindra.com/pages/default.aspx
8/17/2019 Market Guide for Mobile Appl 273008
24/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Altimetrik
www.altimetrik.com
Altimetrik was founded in 2012 and is part of the Vattikuti Ventures. Altimetrik provides business
transformation and technology solutions for enterprises in areas of digital, connected solutions andconsumer technologies through ve capability centers in the following areas, user experience and
design, prototyping, application development and integration, information engineering and
Visualization, quality engineering and automation and DevOps. The provider employs 240 of its
engineers in its mobile application testing practice and draws most of its revenue from North
America, despite its presence in multiple other regions.
Its Quality Engineering Automation Capability Center, is where its mobile testing service resides,
using its delivery centers in India and Uruguay to deliver mobile testing solutions. Altimetrik has a
cloud-based mobile device lab called "Appcentric," a framework that connects to an array of real
mobile devices enabling remote collaboration and test automation. Altimetrik has also developed an
automation framework enabling scriptless, domain-specic test automation called "Framework forEarly Automated Testing" (FEAT). Altimetrik provides engagement models including on-premises,
cloud hosted and subscription-led offerings.
Epam
www.epam.com
Epam is an IT service provider with application development as its core competency.
The company has around 13,600 employees with 2,200 of those based in its testing practice, where
mobile application testing services is part of the portfolio. Most of its consultants are based in
Eastern Europe although it has been increasing its footprint in the North America and Asia/Pacicregions, as part of its efforts to make the company more global. Meanwhile approximately 50% of
its revenue is derived from the U.S. with the rest originating from Europe. A large portion of its
revenue is in mobile application testing, stemming from North America and Western Europe. Both
its testing practice and automation are areas of growth — and its main focus. It has offerings
including mobile test automation and mobile performance testing. Epam offers mobile testing
services as an end-to-end product, a quality engineering service that includes automation solutions,
specialized security testing, mobile performance, usability and user experience assessment.
Competency centers, established methodologies, tools and accelerators support these offerings.
Epam leverages a unied automation framework, which is reusable between Web and mobile in its
testing practice.
Olenick & Associates
www.olenick.com
Established in 1998, Olenick & Associates concentrates on offerings within software development,
including testing. The company primarily focuses on North America with delivery centers in Latin
America and Northern Ireland. Olenick & Associates has created a mobile test lab that it can use to
test applications across multiple mobile devices. It provides functional testing services to clients as
Page 24 of 33 Gartner, Inc. | G00273008
http://www.olenick.com/http://www.epam.com/http://www.olenick.com/http://www.epam.com/http://www.altimetrik.com/
8/17/2019 Market Guide for Mobile Appl 273008
25/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
well as manual testing and test automation. For test automation, it utilizes open-source functional
testing tools; and for mobile user experience testing, the provider has created an internal
framework.
RapidValue Solutionswww.rapidvaluesolutions.com
RapidValue Solutions is a provider focusing on the digital transformation for the enterprise along
three dimensions — mobile, omnichannel and cloud. Having an end-to-end approach is the primary
business, but the provider has established a dedicated sales team bringing in projects specically
for mobile application testing services. RapidValue Solutions employs around 400 resources in total,
of which, 72 are based in the mobile testing practice. The company has ofces in the U.S. and
India; it is looking to expand in the European market by opening its rst ofce in the U.K. The
company follows the scrum development model where the testing team is part of the development
team during the entire project development stages. It also has a dedicated mobile application-
testing center (DMTC) and a dedicated lab with devices, simulators and automation testing tools.The testing portfolio of RapidValue Solutions includes test automation, functional and user
acceptance, performance, usability and security testing.
SLK Software
www.slk-group.com
Part of the SLK Group, SLK Software is a service provider with around 1,500 employees. SLK
Software offers mobile application testing as part of an overall testing service, as well as a discrete
mobile application testing services practice with 50 resources. The company offers testing services
through different engagement models including test automation and performance. SLK Softwareprovides TaaS with an outcome-based pricing model, leveraging its own automation solution and its
own private mobile cloud solution.
StarDust
www.stardust-testing.com
StarDust engages with its clients throughout the process of creating and launching their digital
solutions, including mobile and websites. The mobile application testing practice comprises 60
resources and most of its revenue originates from North America with some clients in EMEA also.
The company provides testing of mobile applications — such as bug tracking or checking if themobile website is responsive to different types of media and devices. It also operates upstream of
the production, both in the design and development phases of project. StarDust works with clients
ensuring that they comply with current technical Web and mobile standards as well as meeting any
functional requirements. The company provides the following types of testing services — functional,
performance, security, location, user experience and automation.
Gartner, Inc. | G00273008 Page 25 of 33
http://www.stardust-testing.com/?lang=enhttp://www.slk-group.com/http://www.rapidvaluesolutions.com/
8/17/2019 Market Guide for Mobile Appl 273008
26/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Syntel
www.syntelinc.com
Syntel is a full service provider including mobile application testing services in its portfolio of
services. Syntel serves clients through its mobile test center of excellence (COE) where it also offerscloud-enabled mobility testing services. Syntel has just over 24,000 employees with 1,300
resources in mobile application testing services. Syntel has developed adaptable test
methodologies for mobile applications and offers both manual and automated testing. Syntel has a
private cloud mobility-testing lab where it leverages its mobility test automation framework (MTAF).
It is built on top of open stack where vendor-specic products can create mobile test automation
suites. It also includes lab-as-a-service (LaaS) into its engagement model providing access to its
private lab. Syntel offerings include functional application testing services, performance testing, test
automation services, user experience testing and security.
Syntel's test lab has multiple devices, emulators and tools. One example is the in-house developed
cross-browser, Test Automation Framework xWAT for testing browser compatibility on both iOS and Android platforms.
Virtusa
www.virtusa.com
Virtusa is an IT services company providing IT consulting, technology and outsourcing services,
with just under 10,000 resources in total. The company was founded in 1996 with ofces and
technology centers throughout the U.S., Europe and Asia. Virtusa uses its global delivery model to
provide services to its clients, which includes testing services. In testing, the company has 148
dedicated resources focusing on mobile application testing services where it utilizes its mobility lab
and test automation frameworks to serve clients. Mobile testing services ts into its managed
service test COE; it also offers a mobile subscription service and a TaaS test factory service for
mobile automation and execution.
Virtusa leverages its test automation platform "centroid," enabling end-to-end digital test
automation.
Pure-Play Testing Service Providers
Pure-play testing service providers gain the vast majority of their revenue from testing services.
These rms can have various specializations, but all include mobile application testing within their
testing portfolio. Buyers can consider these providers for specialist skills, smaller projects, country-specic knowledge or for their exible and nimble approach.
b-acceptance
www.b-acceptance.com
Founded in 2011, b-acceptance is a pure-play testing rm with 35 resources focused on mobile
application testing services; mainly focusing on the digital world (media, m-commerce, e-
Page 26 of 33 Gartner, Inc. | G00273008
http://www.b-acceptance.com/http://www.b-acceptance.com/http://www.virtusa.com/http://www.syntelinc.com/
8/17/2019 Market Guide for Mobile Appl 273008
27/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
commerce). The company's approach concentrates on the validation of business processes.
Consequently, b-acceptance offers services executing test strategy techniques, design scenarios,
recovery of data, test execution and reporting of test campaigns. "TRA Ready Mobile" is a testing
offering that is precongured and immediately available to validate clients' mobile projects.
Cigniti Technologies
www.cigniti.com
Cigniti Technologies is a pure-play testing service provider with a mobile application testing
practice. Cigniti Technologies employs 250 resources that are specialized on mobile application
testing services. Cigniti Technologies offers mobile testing to its clients across U.S., U.K., Australia,
New Zealand, Asia and the Middle East. Lately, this company has grown rapidly, with a value
proposition primarily centered on IP tools and accelerators, but also frameworks, test COE and
investments in global enterprise labs and partnerships. Cigniti Technologies uses its mobile test
COE to combine best practices for its mobile practice and it has just invested in building a next-
generation mobility lab with cloud capabilities. The company's mobility testing services includemobile test automation, mobile performance benchmarking, customer experience and digital
testing, mobile apps testing, functional testing, compatibility testing and security testing for both
enterprises and independent software vendors (ISVs).
CresTech Software Systems
www.crestechglobal.com
CresTech Software Systems is a pure-play testing service provider with a mobile application testing
services practice. The company has over 300 consultants spread across its three delivery centers in
India and North America. CresTech Software Systems has developed a testing platform, "Mobo-Sure," covering all aspects of mobile application quality, including functional, automation,
performance and security testing across various platforms, devices and carriers. Through this
platform, the company offers mobile application testing services including device compatibility
testing, automation and performance as well as security testing services.
CresTech Software Systems has set up a mobile testing lab; enabling multidevice testing that also
includes a patented platform-agnostic test automation framework in its portfolio. CresTech Software
Systems has also developed a reusable test asset library.
Kualitatem
www.kualitatem.com
Kualitatem is an independent software testing and information systems pure-play company with
ofces in the U.S. and United Arab Emirates (UAE). It provides services for Web, mobile and
desktop applications and its mobile application testing services practice includes 35 resources in
total. Kualitatem has developed a platform for mobile application testing covering key testing types
Gartner, Inc. | G00273008 Page 27 of 33
http://www.kualitatem.com/http://www.cigniti.com/http://www.kualitatem.com/http://crestechglobal.com/http://www.cigniti.com/
8/17/2019 Market Guide for Mobile Appl 273008
28/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
such as functional, security and performance, compatibility, usability and localization testing. The
company has a lab where it provides access to customers while testing using real-time devices.
QA Mentor
www.qamentor.com
QA Mentor is a pure-play testing services company established in 2010. Mobile testing is part of its
"Lab Compatibility Services," which is also offered separately to clients. Although all 106 company
resources are able to support mobile testing projects, there are 28 dedicated resources focused on
just mobile testing. Most of the mobile practice revenue is generated in North America. Despite this,
revenue is also derived from Europe (France, Germany, Ukraine) and Russia, as well as India. QA
Mentor uses its global delivery centers to provide services to clients of varying size and scale using
its own methodology. The company offers mobile testing services along with both functional and
nonfunctional areas of testing. An e-commerce platform is what the company considers to be one
of its key "sweet spots."
QASource
www.qasource.com
QASource employs around 550 QA engineers. QASource works with clients in North America with
delivery provided by its Indian delivery center. This pure-play testing service provider offers mobile
application testing services as part of its portfolio. Depending on client requirements and growth,
the number of mobile testers uctuates slightly, but around 240 resources are dedicated to the
mobile application testing practice. QASource has some local presence in the shape of project
managers in North America but the rest of its team is based in India. QASource has a mobile device
lab of over 150 real devices utilized by its experienced test engineers.
QualiTest Group
www.qualitestgroup.com
QualiTest Group is the second largest pure-play testing service provider globally. Most of its revenue
originates from its operations in North America, Israel, U.K., the Netherlands, India and Australia.
The total application testing resources of QualiTest Group comprise 2,000 resources — and within
this practice it also offers mobile testing services. Mobile test automation is a differentiator and it
provides coverage on all major devices and platforms.
The company also uses mobile test labs where they can provide access to environments for
distributed testing. QualiTest Group aims for exibility with tools that can be generated quickly to
get projects running as well as by offering on-demand services. QualiTest Group provides testing
through the life cycle of consulting to managed services as well as crowdtesting services. Services
include tests such as test process improvement, security, accessibility and load testing.
Page 28 of 33 Gartner, Inc. | G00273008
http://www.qualitestgroup.com/https://www.qasource.com/http://www.qamentor.com/
8/17/2019 Market Guide for Mobile Appl 273008
29/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Testing4Success
www.testing4success.com
Testing4Success is a pure-play testing service provider focusing on usability testing services. The
company ensures quality on Web, mobile or desktop applications. It has built QA test labscontaining hardware, software and tools to create multiple test environment. Testing4Success offers
mobile-application testing services, including testing on real devices. The services offered by
Testing4Success include functional testing, compatibility testing and usability testing.
ZenQ
www.zenq.com
Created in 2003, ZenQ is a pure-play testing services company with around 300 resources, with 75
residing in the mobile application testing services practice. ZenQ works with clients in the U.S.,
U.K., Canada and other parts of Europe (via its ofces in the U.S. and India). It organizes its mobile
application-testing offering along functional, performance, security and automation testing services.
It has COEs for major testing functional areas. ZenQ has developed expertise across mobile OS
platforms, offers device-agnostic testing, testing on mobile simulators and mobile platforms, and
testing automation using commercial and open-source tools.
Mobile Service Providers
Mobile service providers have mobility as their core focus. Buyers can consider these providers for
their specialist mobile skills and focus, often using modern approaches.
InfoStretchwww.infostretch.com
InfoStretch is a pure-play mobile testing provider of next-generation mobile application life cycle
services, ranging from design and development to testing and sustenance. The company employs
around 700 resources and is focused on the U.S. market, but has also expanded into the European
marketplace. Since 2004, it has been offering solutions and services to enterprises in the mobile
ecosystem. The company uses its two mobile development centers to support delivery as well as a
test lab with access to various OS and devices. It develops accelerators on top of commercial
existing tools or open-source tools and has developed automated tests for mobile handset device
testing. The range of services offered by the company includes functional, nonfunctional,
performance testing, usability testing, user experience, language testing and security testingservices.
Market RecommendationsMobile testing is no longer simply about the functionality of an application. It must cover all the
aspects of platforms, networks and devices in order to meet user expectations, which require very
Gartner, Inc. | G00273008 Page 29 of 33
http://www.infostretch.com/http://zenq.com/http://www.testing4success.com/
8/17/2019 Market Guide for Mobile Appl 273008
30/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
specic skills from providers. Evaluate mobile testing service providers based on all the aspects of
complex mobile testing requirements such as infrastructure, skills, tools, performance, usability and
end-to-end quality approach and strategy.
In addition, understand the specic demands of the organization and the objectives they are trying
to achieve, which need to account for aspects such as variations in the development methods andprocesses that must be supported (waterfall, agile, mixed), support of bimodal, partnerships and
availability of tools, accelerators and IP.
Page 30 of 33 Gartner, Inc. | G00273008
8/17/2019 Market Guide for Mobile Appl 273008
31/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Acronym Key and Glossary Terms
API application programming interface
Benelux Belgium, the Netherlands and Luxembourg
CIO chief information ofcer
COE center of excellence
EMEA Europe, the Middle East and Africa
iOS iPhone operating system
IoT Internet of Things
IP intellectual property
ISV independent software vendor
LaaS lab-as-a-service
MTAF mobility test automation framework
NFC Near Field Communication
OS operating system
POC proof of concept
POS point of sale
QA quality assurance
RFI request for information
RFP request for proposal
TaaS testing-as-a-service
TCS Tata Consultancy Services
UAE United Arab Emirates
UI user interface
UX user experience
Gartner, Inc. | G00273008 Page 31 of 33
8/17/2019 Market Guide for Mobile Appl 273008
32/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
Gartner Recommended ReadingSome documents may not be available as part of your current Gartner subscription.
"Market Guide for Crowdsourced Application Testing Services"
"Market Guide for Mobile Application Testing"
"How to Choose a Mobile Application Testing Service Provider"
"Magic Quadrant for Application Testing Services, Worldwide"
"Essential Metrics for Outsourcing Testing Services"
Evidence
"Flipping to Digital Leadership: The 2015 CIO Agenda"
1 Gartner analysts received 47 inquiries on mobile application testing services between September
2013 through September 2014, with an increase between September 2014 through September
2015 of 64.
Page 32 of 33 Gartner, Inc. | G00273008
8/17/2019 Market Guide for Mobile Appl 273008
33/33
This research note is restricted to the personal use of dan.mcfall@mobilelabsinc.com
GARTNER HEADQUARTERS
Corporate Headquarters
56 Top Gallant RoadStamford, CT 06902-7700
USA +1 203 964 0096
Regional Headquarters
AUSTRALIA BRAZILJAPANUNITED KINGDOM
For a complete list of worldwide locations,visit http://www.gartner.com/technology/about.jsp
© 2015 Gartner, Inc. and/or its afliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. or its afliates. This
publication may not be reproduced or distributed in any form without Gartner’s prior written permission. I f you are authorized to accessthis publication, your use of it is subject to the Usage Guidelines for Gartner Services posted on gartner.com. The information contained
in this publication has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy,completeness or adequacy of such information and shall have no liability for errors, omissions or inadequacies in such information. This
publication consists of the opinions of Gartner’s research organization and should not be construed as statements of fact. The opinions
expressed herein are subject to change without notice. Although Gartner research may include a discussion of related legal issues,Gartner does not provide legal advice or services and its research should not be construed or used as such. Gartner is a public company,
and its shareholders may include rms and funds that have nancial interests in entities covered in Gartner research. Gartner’s Board of
Directors may include senior managers of these rms or funds. Gartner research is produced independently by its research organizationwithout input or inuence from these rms, funds or their managers. For further information on the independence and integrity of Gartner
research see “Guiding Principles on Independence and Objectivity ”
http://www.gartner.com/technology/about/ombudsman/omb_guide2.jsphttp://www.gartner.com/technology/about/ombudsman/omb_guide2.jsphttp://www.gartner.com/technology/about/policies/usage_guidelines.jsphttp://www.gartner.com/technology/about.jsp