Critical Capabilities for Network Performance …...improved network capacity management, thereby...

Gartner Reprint

file:///C/Users/kieran.rowlinson.PC-153650147731/Documents/Gartner%20Critical%20Capabilities%202018.htm[14/03/2018 10:02:57]

LICENSED FOR DISTRIBUTION

Critical Capabilities for Network Performance Monitoring and Diagnostics

Published: 21 February 2018 ID: G00326771Analyst(s): Vivek Bhalla, Sanjit Ganguli, Pankaj Prasad

Summary

NPMD tools enable organizations to measure and improve IT services. To choose the correct tool, I&O leadersresponsible for network performance should determine which of the popular NPMD products are best-suitedto their requirements and maturity level.

Overview

Key Findings

The profiles of key users of network performance monitoring and diagnostics (NPMD) tools are diversifying,with three out of five primary stakeholders residing outside of network operations.

NPMD tools are increasingly positioned to provide insight into the usage of applications and the activities ofend users, including their impact on business objectives.

Navigational search, contextual workflow and analytics-based functionalities continue to become morebroadly available in NPMD tools, but their degree of sophistication varies widely among vendors.

An increasing number of vendors focus on non-NPMD functions, such as infrastructure availabilitymonitoring and network fault management, at the expense of the core performance capabilities that mostI&O organizations require.

Recommendations

To optimize IT operations and DevOps to drive business value, I&O leaders should:

Ensure solutions are applicable beyond a NetOps 2.0 context by collaborating with stakeholders and expertsfrom outside I&O when evaluating an NPMD tool.

Support bandwidth-intensive services (such as VoIP and UCM) by deploying NPMD technology to monitornetwork demand and manage capacity planning.

Cultivate collaboration across IT operations by using the analytics and contextual user interface drill-downworkflows embedded in NPMD tools to share data across domain groups.

Strike a cost-effective balance by using packet-based inspection for mission-critical, granular insight, as wellas flow collection techniques for holistic, infrastructurewide coverage.

Strategic Planning Assumption

https://www.gartner.com/home

Gartner Reprint


By 2023, 50% of buyers of NPMD tools will reside outside traditional network operations organizations,increasing from 20% in 2018.

What You Need to Know

This document was revised on 21 February 2018. The document you are viewing is the corrected version. Formore information, see the Corrections page on gartner.com.

NPMD tools support many roles within IT and, increasingly, also support business initiatives. Measuring andreporting on network performance is crucial to ensure that the proper steps are taken so performance stays atan acceptable level. It can also highlight opportunities to improve business outcomes for internal end usersand external customers through improved network and application delivery. Finally, NPMD tools enableimproved network capacity management, thereby lowering capital investment for networking equipment andservices.

Gartner has identified five key capabilities for NPMD tools: endpoint, component and link monitoring; servicedelivery monitoring; diagnostics; analytics; and integration and interoperability. These capabilities are critical tothe success of five different types of stakeholders commonly involved in this endeavor: network operator,network architect, IT operations manager, IT operations generalist and the line-of-business user. Note howthree of the five stakeholders are roles beyond traditional network operations. These capabilities serve as theprimary basis of this analysis and together play a key role in network and IT operations. This research focuseson the competitiveness of NPMD solutions themselves, as opposed to the strengths and cautions of thesupplying vendors that are covered in the companion Magic Quadrant research. Readers can use this CriticalCapabilities research and role-centric perspective to inform their own analysis and determine the best fit fortheir objectives and organization.

This analysis complements the "Magic Quadrant for Network Performance Monitoring and Diagnostics," whichdefines the market and highlights a broad set of factors, including the corporate viability, vision, marketing andgeographic focus of the vendors that offer these NPMD products. We strongly recommend that organizationsuse this research in conjunction with the Magic Quadrant, inquiries with Gartner analysts and other Gartnerresearch to define their requirements and select the solutions that match their needs.

Analysis

Critical Capabilities Use-Case Graphics

Figure 1. Vendors' Product Scores for the Network Operator Use Case

http://www.gartner.com/technology/about/policies/current_corrections.jsp

Gartner Reprint


Source: Gartner (February 2018)

Figure 2. Vendors' Product Scores for the Network Architect Use Case

Gartner Reprint



Figure 3. Vendors' Product Scores for the IT Operations Generalist Use Case

Gartner Reprint



Figure 4. Vendors' Product Scores for the IT Operations Manager Use Case

Gartner Reprint



Figure 5. Vendors' Product Scores for the Line-of-Business User Use Case

Gartner Reprint



Vendors

Accedian (Performance Vision) Performance Vision 4.2Founded in 2004, Performance Vision is a privately owned subsidiary of Accedian. Performance Vision isheadquartered in Paris, France. Performance Vision's NPMD offering, Performance Vision v4.2, was evaluatedfor this research. This product is an on-premises hardware and software offering.

Performance Vision is delivered as a lightweight capture appliance and a data store appliance. The captureappliance is focused on data collection, traffic analysis and sending metrics to the data store appliance. Thedata store appliance provides the data visualization and reporting capabilities, while also assisting with datacollection. Performance Vision demonstrates granularity and performance at scale, with significant attention tothe visualization of data to facilitate easier diagnosis of performance anomalies. Performance Vision relies onpassive data collection methods for monitoring. The tool provides transaction visibility across webapplications, databases, name services, file transfer and storage, and VoIP. The emphasis of the previous

Gartner Reprint


release focused on scalability and distributed environments. The latest release, in April 2017, delivered furthersupport for a distributed deployment model with the introduction of a new hardware microappliance, designedfor smaller remote sites. Additional enhancements include support for Citrix performance analysis andNexthink integration.

Gartner's survey of end users rated Performance Vision strongly for its ease of implementation and upgrade,and maintenance and reliability. The same survey rated Performance Vision poorly in terms of integration withthird-party tools and analytics. When asked what other vendors were considered when selecting their NPMDsolution, Performance Vision end users cited SolarWinds most frequently, followed by Riverbed.

This product is well-suited for midsize enterprises that require a small deployment model footprint withvisibility across applications and the network layers. Based on Gartner's analysis, the product is best aligned tothe line-of-business user role.

Note: Performance Vision was acquired by Accedian in December 2017. The change in ownership occurredafter the analysis conducted for this research took place.

AppNeta Performance Manager 10.8Founded in 2011, AppNeta is a privately owned company headquartered in Boston, Massachusetts. AppNeta'sNPMD product, AppNeta Performance Manager 10.8, was evaluated for this research. This product is a SaaSand on-premises offering.

AppNeta Performance Manager consists of three components: delivery, experience and usage. Together, thesecomponents have several functions, including synthetic transaction monitoring that has been developed tomonitor SaaS-based applications in addition to traditional on-premises applications. Also included is pathanalytics, employing active monitoring and diagnostics. Finally, application usage analytics is done using deeppacket inspection capabilities. Enabling these use cases are physical, virtual or software monitoring points thatmust be installed at each remote location. Preconfigured VM and software monitoring points can bedownloaded, or preconfigured hardware can be supplied by AppNeta. Pricing has been simplified compared toearlier offerings, with customers paying an annual fee per monitoring application and location, with additionalone-time charges for monitoring points.

AppNeta's NPMD solution focuses on ease of implementation, low total cost of ownership and visibility ofapplications irrespective of where they may be deployed or hosted. The solution is a modular public- andprivate-cloud-hosted service, with the majority of end users on the public cloud offering.

Gartner's survey of end users rated AppNeta among the highest in terms of ease of upgrading as well asendpoint, component and link monitoring. The same survey indicated that AppNeta was rated poorly in termsof satisfaction with the cost of maintenance and integration with third-party tools. When asked what othervendors were considered when selecting their NPMD solution, AppNeta end users cited Riverbed andSolarWinds.

This product is well-suited for highly distributed enterprises, or those organizations with a significant number ofapplications deployed in cloud or hybrid environments. Based on Gartner's analysis, the product is best alignedto the line-of-business user role.

CA Technologies Network Operations and AnalyticsFounded in 1976, CA Technologies is a public company headquartered in New York, New York. The CA

Gartner Reprint


Technologies NPMD product, Network Operations and Analytics v3.2, was evaluated for this research. Thisproduct is an on-premises software offering.

CA Technologies' product suite has been built through both acquisition and organic development, and iscurrently being offered as part of the Network Operations and Analytics platform. This is an attempt to unifythe NPMD solution into a single portal that joins data from multiple sources. The Network Operations andAnalytics platform is based on CA Performance Management as its foundation.

The Network Operations and Analytics platform is composed of CA Performance Management forinfrastructure monitoring, CA Network Flow Analysis for traffic monitoring, CA Unified CommunicationsMonitor for voice and video monitoring, and CA Application Delivery Analysis, which is focused on packet-levelinspection for applications. This is rounded out by CA Virtual Network Assurance, for the monitoring andmanagement of modern network architectures. New to CA's portfolio is the SaaS-based Digital ExperienceInsight solution, which is a full-stack monitoring and analytics service offering that provides analysis acrosscustomer experience, application performance, and infrastructure and network management. Support forNPMD workflows remains limited to date, but additional integration is on the roadmap. Gartner has observedsome overlapping capabilities across products and shifting product and integration roadmaps.

Gartner's survey of end users indicated the company's ability to commit and deliver requested enhancementswas above average, and also saw a marked increase in user satisfaction of CA's peer user community over theprevious year. End users rate business transaction monitoring and ease of integration and deployment amongthe weakest. When asked what other vendors were considered when selecting their NPMD solution, CATechnologies' end users cited Riverbed and SolarWinds most frequently, followed equally by Cisco, LiveActionand NetBrain.

This product is well-suited for large enterprises and service providers that require multitenancy and a focus onflow and Simple Network Management Protocol (SNMP) monitoring. Based on Gartner's analysis, the productis best aligned to the network operator role.

Cisco Prime Infrastructure, NAM and Tetration AnalyticsFounded in 1984, Cisco is a public company headquartered in San Jose, California. Cisco's NPMD products(Prime Infrastructure v3.2, APIC-EM 1.5, Network Analysis Module [NAM] v6.2.1, and Tetration Analyticsv2.0.2.20) were evaluated for this research. The offering is composed of on-premises software and hardware.

Prime Infrastructure has multiple capabilities spanning network configuration, monitoring and themanagement of Cisco equipment, with comprehensive support for wireless network monitoring. This iscoupled with the Application Policy Infrastructure Controller Enterprise Module (APIC-EM) capability, whichprovides path and performance visibility and a central control point for iWAN. NAM is a hardware appliance ormodule that can be included in many Cisco network devices for handling deep packet inspection (DPI).Tetration Analytics is designed to address data center operations and security-related use cases, usinghardware-embedded network telemetry.

New to Cisco's offering is DNA Center 1.0, which aims to deliver a single architecture for managing the LAN,WAN, Campus and Cloud for assurance and automation via an intent-based networking approach. The firstsolutions to ship on DNA Center (Software Defined Access [SDA] and Network Assurance) aim to bringmachine learning and analytics capabilities through the ingestion of a broad set of data sources from thenetwork environment.

Gartner Reprint


Gartner's survey noted that Cisco's NPMD solution was frequently considered across all end users,demonstrating brand awareness and profile. The same survey indicated Cisco's NPMD solution was ratedamong the lowest for value for cost of implementation and maintenance, and integration with third-party tools.When asked what other vendors were considered when selecting their NPMD solution, Cisco end users citedSolarWinds most frequently, followed by Micro Focus (HPE Software) and Riverbed.

This product is well-suited for enterprises with a heavy investment in Cisco hardware, especially wireless, thatwant to monitor performance. Based on Gartner's analysis, the product is best aligned to the network architectrole.

Colasoft UPM, nChronos and CapsaFounded in 2001, Colasoft is a privately owned company headquartered in Chengdu, China. Colasoft's NPMDproducts (UPM v5.2, nChronos v5.3 and Capsa v10.0) were evaluated for this research. The offering iscomposed of on-premises software and hardware.

Colasoft UPM delivers a visualization layer that provides business- and service-oriented performance views, fullnetwork path analysis, and proactive performance issue detection, all based upon data collected by nChronos.nChronos provides packet capture and historical and real-time analysis of over 80 application-orientedperformance metrics for every TCP conversation. Capsa is a network analyzer that provides packet sniffingand capture where application decoding and deep domain analysis are required.

All metrics can be collected and reported at a one-second granularity rate, a differentiator compared to mostother NPMD vendors. Statistical data compressing storage and packet indexing technology enable rapid drill-down, retrieval and downloading of data. Automatic multisegment analysis is another capability of the solution.Colasoft supports the monitoring of virtual machines and software-defined networking (SDN) infrastructures.UPM also offers user-defined alarming based upon a three-dimensional triangulation across application, serverand network domains. However, abnormal business alarms and analysis require manual configuration.

Gartner's survey of end users scored Colasoft strongest in terms of its ability to commit and deliver onrequested enhancements and navigation and workflow, and it is tied with LiveAction for strongest in terms ofcontextual drill-down and troubleshooting. The same survey rated Colasoft below the median in terms of costof maintenance. When asked what other vendors were considered when selecting their NPMD solution,Colasoft end users cited NETSCOUT and Riverbed most frequently, followed by SolarWinds.

These products are well-suited for service providers and large enterprises, particularly governmentorganizations, needing to combine packet analysis with multimetric alarm and threshold alerting. Based onGartner's analysis, the product is best aligned to the network operator role.

Corvil Analytics 9.2Founded in 2000, Corvil is a privately owned company headquartered in Dublin, Ireland. Corvil's NPMD product,Corvil Analytics v9.2, was evaluated for this research. This product is an on-premises appliance offering.

Corvil Analytics is a competitively priced, packet-based NPMD tool that focuses on analyzing high volumes ofin-flight wire data. The latest release includes a focus on improved user interface and workflows, particularly interms of VoIP and application dependency visualization. Automatic comparison and highlighting of keyperformance indicators has also been added, enabling users to define comparison periods and flag deviationfrom previous periods for any performance indicator. Integration with third-party systems, including Splunk,

Gartner Reprint


LiveAction and Cisco Tetration, has also been added. Limited support of VDI protocols, such as Citrix's ICA aswell as cloud and virtual environments, continues to be an inhibitor to adoption.

Gartner's survey of end users scored Corvil strongly in terms of direction and vision. The same survey ratedCorvil among the weakest in terms of cost and ease of implementation. When asked what other vendors wereconsidered when selecting their NPMD solution, Corvil end users cited NETSCOUT most frequently, followedby Riverbed.

The product is well-suited for enterprises in need of short-time scale and high granularity network monitoring,such as in the electronic trading market and other digital businesses. Based on Gartner's analysis, the productis best aligned to the line-of-business user role.

ExtraHop Platform 7.0Founded in 2007, ExtraHop is a privately owned company headquartered in Seattle, Washington. ExtraHop'sNPMD product, Extrahop Platform v7.0, was evaluated for this research. The offering is composed of on-premises software, hardware, additional virtual appliances and SaaS components.

The ExtraHop Platform consists of ExtraHop Discover, Explore, Command and Trace Appliances alongside anew SaaS offering, ExtraHop Addy. The platform focuses on wire data analytics, providing deep packetinspection and real-time transaction analysis capabilities at scale for packets traversing the network. This deeppacket inspection allows for visibility into a wide range of application data, both at the header and payload level.ExtraHop has expanded its offering with Addy, a SaaS-based, unsupervised machine learning feature forNPMD data, focused on anomaly detection. This is coupled with an Open Data Stream initiative that allows thisanomaly detection engine to integrate with third-party products, including Cisco Tetration, ServiceNow andSplunk. ExtraHop has also improved its network dependency visualization with live activity maps, although thisis limited to traffic only and does not map layer 2/3 topology.

Gartner's survey of end users rated ExtraHop's solutions strongly in terms of direction, vision, and the ability tocommit and deliver requested enhancements. The same survey indicated ExtraHop's solution ranked lowamong vendors that were also considered in NPMD procurements, and remains below average for endpointand component monitoring. When asked what other vendors were considered when selecting their NPMDsolution, ExtraHop end users cited Riverbed most frequently, followed by SolarWinds and Cisco.

The ExtraHop solution is well-suited for enterprises in need of application-aware network performancemonitoring to supplement existing infrastructure monitoring, particularly where application decode support is arequirement. Based on Gartner's analysis, the product is best aligned to the line-of-business user role.

Flowmon Networks 9.0Founded in 2007, Flowmon Networks is a privately owned company headquartered in Brno, Czech Republic.Flowmon Networks' NPMD offering, Flowmon v9.0, was evaluated for this research. This product is an on-premises hardware, virtual appliance and SaaS offering.

Flowmon is a cost-effective, flow-oriented NPMD tool that looks to meet the most demanding of performanceand scalability requirements. The vendor was one of the earliest to support 100G environments with its100GbE Flowmon probe. The tool can process and analyze over 400,000 flows per second per appliance.Packet support remains limited to flow conversions, as opposed to detailed packet analysis, but trigger-basedpacket capture on event detection is supported. In its latest release, Flowmon has extended support for Layer 7

Gartner Reprint


protocols, including SMTP, IMAP, POP3, MSSQL and HTTP. Improving the user interface remains a priority forFlowmon, which is currently disjointed between different workflows and relies too heavily on manual filter andreport creation.

Gartner's survey of end users rated Flowmon among the highest in terms of ease of implementation andupgrading, and the ability to contextually drill down for troubleshooting. The same survey rated Flowmon asweak in terms of integration, interoperability and the quality of its peer user community. When asked whatother vendors were considered when selecting their NPMD solution, Flowmon end users cited SolarWindsmost frequently, followed by Cisco.

This product is well-suited for midmarket and large enterprises and service providers that need a scalable flow-oriented NPMD solution with diverse deployment options and an ability to meet security use cases. Based onGartner's analysis, the product is best aligned to the network operator role.

H3C iMC 7.3Founded in 2003, H3C is a privately owned company headquartered in Beijing and Hangzhou, China. Its NPMDoffering, intelligent Management Center (iMC) v.7.3, was evaluated for this research. iMC is an on-premisessoftware offering.

H3C iMC has two editions — the standard and the professional edition. The standard version of iMC supportsan embedded database with a license limit of 15,000 nodes. The professional edition is based around the useof an external database of the end user's choosing. The basic network monitoring capabilities can be extendedthrough the use of modules, such as network traffic manager (NTM), virtual connection manager (VCM), virtualfabric manager (VFM) and wireless service manager (WSM). iMC also supports unified communicationsmonitoring (UCM) through its ability to monitor latency, jitter and packet loss. The product also providesadditional infrastructure monitoring modules for all major virtual server environments, SAN fabric and aselection of open-source databases. The current release extends display support for devices from OEMs (forexample, virtual device contexts [VDCs] for Cisco, link displays for Avaya devices, and the enablement ofNetStream as a data source for Huawei devices). The current version also adds support to select lower-LevelNMS so they can be displayed in their 3D data center topology simulation.

Gartner's survey of end users rated H3C strongly in terms of the ability to commit and delivery on requestedenhancements, as well as for the ease of maintenance and reliability of the tool. The same survey rated ease ofupgrade, business transaction monitoring and analytics below the median. When asked what other vendorswere considered when selecting their NPMD solution, H3C end users cited Cisco most frequently, followed bySolarWinds.

This product is well-suited for SMB, midmarket and managed service providers that need a competitivelypriced option and prefer device-centric monitoring. Based on Gartner's analysis, the product is best aligned tothe network operator role.

InfoVista VistaInsight for Networks, VistaGo and 5View SuiteFounded in 1995, InfoVista is a privately owned company headquartered in Paris, France. InfoVista's NPMDofferings (VistaInsight for Networks v5.6, VistaGo v1.0 and the 5View suite of products) were evaluated for thisresearch. These products are an on-premises software and managed service offering.

The products can be scaled up and deployed depending on what level of visibility is needed, including SNMP,

Gartner Reprint


flow and packet data. Each of the products within the offering collects data and, based on user-definedconfiguration, rolls data up incrementally, allowing for large scale. Each part of the offering is multitenant andallows for multiple users within a single master instance of the solution. This architecture, although scalable,comes with complexity, requiring a service engagement to deploy. End users typically use VistaInsight, whichprovides a reporting interface to pull usage and performance reports. Due to the number of products, userinterfaces and implementation nuances, Gartner has observed that InfoVista caters toward and becomesembedded in large carrier customers.

In February 2017, InfoVista launched VistaGo, a cloud-based managed service offering, targeted at smallcompanies (mainly CSPs and MNOs) to provide customer-facing performance visibility. The latest release ofVistaInsight for Networks in September 2017 adds IP end-to-end measurement support through Two-WayActive Measurement Protocol (TWAMP), allowing for Loss Measurements (LM) and Delay Measurements(DM). Support has also been enhanced for vendors like RAD, Ciena, Bluecoat, Huawei and Accedian. Over thelast 12 months, VLAN grouping and naming has been included in 5View applications toward improvement oftraffic analysis, while site audit report was included in 5View Service Data Manager.

Gartner's survey of end users scored InfoVista among the strongest vendors in terms of historical reportingand exporting. The same survey indicated InfoVista end users spend more, as a percentage, on totalimplementation, training and professional services costs, with ease of upgrading being a challenge comparedto most other NPMD vendors. End users also rated InfoVista poorest in terms of cost of maintenance. Whenasked what other vendors were considered when selecting their NPMD solution, InfoVista end users cited CAand Micro Focus (HPE) most frequently.

This product is well-suited for large carriers that need diverse deployment options, multitenancy and extensivereporting out of their NPMD investments. Based on Gartner's analysis, the product is best aligned to thenetwork architect role.

InfoVista declined the opportunity to provide a product demonstration as part of the research process that wasoffered to participating vendors for this research. Gartner's analysis for this vendor is based on other crediblesources, including a completed vendor survey response, previous vendor briefings and interactions, thevendor's own marketing collateral, public information, and discussions with end users.

Ipswitch WhatsUp Gold 2017 PlusFounded in 1991, Ipswitch is a privately owned company headquartered in Burlington, Massachusetts.Ipswitch's NPMD product, WhatsUp Gold (WUG) 2017 Plus, was evaluated for this research. This product is anon-premises software offering.

WUG is offered in Basic, Pro, Total or Total Plus editions, reflecting different assortments of capabilities that aredelivered through an integrated single user interface. These capabilities include Wireless Monitoring, VirtualMonitoring, Application Monitoring, Network Traffic Analysis and Configuration Management. WUG is largelyfocused on IT infrastructure monitoring workflows, with basic packet ingestion and flow conversioncapabilities. Packet data is handled via a packet-to-flow converter agent called Flow Publisher. Improvementsto the solution, in the July 2017 release, include active monitoring of AWS and Azure, as well as support forEMC unity storage. Recent enhancements also include support for distributed deployments to enabledecentralized monitoring, and high-availability and fine-grained granularity polling for up to 20 devices at a time,including updates every two seconds. From a reporting perspective, the latest version adds out-of-boxreporting for top ports and top protocols to its already existing network traffic portfolio of out-of-box reports.

Gartner Reprint


Gartner's survey of end users rated Ipswitch's solutions strongly in terms of satisfaction based on productcost. The same survey rated Ipswitch's solution poorly for integration with third-party tools, and for service andsupport. When asked what other vendors were considered when selecting their NPMD solution, Ipswitch endusers cited SolarWinds most frequently, followed by Micro Focus (HPE), CA and Cisco.

This product is well-suited for small and midsize enterprises that require an IT infrastructure monitoringsolution with the ability to expand with network traffic analysis. Based on Gartner's analysis, the product is bestaligned to the network architect role.

LiveAction LiveNX 6.2Founded in 2007, LiveAction is a privately owned company headquartered in Palo Alto, California. LiveAction'sNPMD product, LiveNX v6.2, was evaluated for this research. The product is an on-premises software offering.

LiveNX is focused on visibility and analytics to help simplify network management. An additional component,LiveUX delivers end-user experience monitoring. For non-Cisco environments, LiveAction uses LiveSensor forpacket inspection and performance metric collection in locations where NetFlow, or other network devicetelemetry data, is not available. LiveNX can collect and store over 1.05 million flows per second via a singlecollector and supports a distributed architecture.

The product development cadence continues to be healthy with significant updates in April 2017 and July2017. The April release included simplified workflows for SD-WAN troubleshooting, integration with Cisco APIC-EM and support for cloud licensing, with monthly usage and billing for large MSP deployments. The Julyrelease saw the launch of LiveInsight — LiveAction's machine-learning-driven analytics — and an expansion ofdevice support to 107 networking vendors. Pricing and a predominant focus on support of Cisco infrastructurecontinue to be inhibitors to adoption.

Gartner's survey of end users rated LiveAction strongest in terms of ease of implementation, historicalreporting and exporting, and it is tied with Colasoft for strongest in terms of diagnostics capabilities.LiveAction's ability to commit and deliver on requested enhancements, maintenance and reliability, and itscontextual drill-down and troubleshooting, also scored strongly. The same survey rated LiveAction poorly interms of ease of upgrading. When asked what other vendors were considered when selecting their NPMDsolution, LiveAction end users cited Cisco most frequently, followed by SolarWinds.

This product is well-suited for enterprises seeking an NPMD tool with a strong emphasis on visual analysis andtroubleshooting, particularly those with a heavy investment in Cisco hardware or SD-WAN. Based on Gartner'sanalysis, the product is best aligned to the IT operations generalist role.

ManageEngine OpManager 12.3Founded in 1996, ManageEngine is a private company headquartered in Pleasanton, California.ManageEngine's NPMD product, OpManager v12.3, was evaluated for this research. This product is an on-premises software offering.

OpManager is an integrated network management platform that provides network performance monitoring,packet inspection, physical and virtual server monitoring, flow-based bandwidth analysis, and additionalcapabilities beyond the scope of this report. Version 12 of OpManager was released in February 2016, with afocus on tight integration of the modules that can be purchased individually but folded into the holistic platformwith minimal overhead. This version also delivered an overhaul of the user interface and more streamlined

Gartner Reprint


upgrade and backup process. In February 2017, DPI support was added to the NetFlow Analyzer componentalong with support for Meraki devices. In August 2017, DPI was fully incorporated into the latest version (12.3).

Gartner's survey of end users scored ManageEngine strongly in terms of integration and interoperability, costof implementation, and cost of maintenance. The same survey rated ManageEngine among the weakest interms of maintenance and reliability, direction and vision, contextual drill-down and troubleshooting, andhistorical reporting and exporting. When asked what other vendors were considered when selecting theirNPMD solution, ManageEngine end users cited SolarWinds most frequently, followed by CA.

This product is well-suited for small and midsize enterprises that require an IT infrastructure monitoringsolution with the ability to expand into network traffic analysis. Based on Gartner's analysis, the product is bestaligned to the network architect role.

Micro Focus NNMi and RUMFounded in 1976, Micro Focus (HPE) is a public company headquartered in Newbury, United Kingdom. MicroFocus' NPMD products (Network Node Manager i [NNMi] v10.21 and Real User Monitoring [RUM] v9.40) wereevaluated for this research. These products are an on-premises software offering.

Designed for network operations specialists and architects and reflecting its predominant use, the Micro Focus(HPE) solution flexibly supports service delivery monitoring and diagnosis, informed primarily by SNMP andflow data. Packet inspection and additional application context gathering to support complex diagnosis isprovided through a data interchange, modeling and process-level integration with Micro Focus's RUM product.Enabling application mapping requires users include the NNMi Smart Plug-In Performance for Traffic for flow-based traffic mapping. Given the multiple component products, this results in two distinctly different UIs. Bothproducts were acquired by Micro Focus as part of the merger/spin-off with HPE Software that took place inSeptember 2017.

Over the last 12 months, Gartner has observed incremental updates. The latest version of NNMi v10.30 wasreleased in June 2017 and delivered support for BGP session monitoring and reporting, enhancements aroundMPLS support, and scalability improvements. The complexity of new deployments and a reliance on additionalcomponents beyond NNMi and RUM to maximize the product's value continue to inhibit adoption.

Gartner has observed that end users continue to indicate they use Micro Focus (HPE) NPMD tools mostfrequently to perform real-time monitoring and alerting. The cost of implementation, ease of implementation,and ability to commit and deliver requested enhancements are areas that end users continue to cite as belowexpected levels. Gartner has observed that Micro Focus (HPE) end users frequently shortlist CA andSolarWinds as additional vendors for consideration when selecting their NPMD solution.

This product is well-suited for large enterprises with deep and varied NPMD requirements, especially those withsignificant Micro Focus ITOM tooling investments. Based on Gartner's analysis, the product is best aligned tothe network operator role.

Micro Focus (HPE) did not respond to requests for supplemental information and/or engage in Gartner'sstandard procedures to address the contents of this document. Gartner's analysis for this vendor is based onother credible sources, including previous vendor briefings and interactions, the vendor's own marketingcollateral, public information, and discussions with end users.

NETSCOUT nGeniusONE and nGeniusPULSE

Gartner Reprint


Founded in 1984, NETSCOUT is a public company headquartered in Westford, Massachusetts. NETSCOUT'sNPMD products (nGeniusONE, InfiniStream, TruView and nGeniusPULSE) were evaluated for this research.The offering is composed of on-premises appliance, software and SaaS products.

NETSCOUT's focus on supporting a broad range of diagnostic needs of network specialists has resulted in adiverse set of hardware appliances and software that is managed through the nGeniusONE platform. FeedingnGeniusONE is the InfiniStream platform, which gathers and stores traffic data from the network and enablestroubleshooting and forensic workflows. nGeniusPULSE provides synthetic testing focused on verifying theperformance of cloud applications. The Fluke-heritage Truview appliance provides unified networkperformance management with application awareness, and overlaps with nGeniusONE.

To address the increasing prevalence of cloud-hosted workloads, NETSCOUT introduced vStream and vScoutfor software-based instrumentation in cloud environments. This integrates with NETSCOUT's nGeniousONEplatform for a combined view of data center and cloud traffic. NETSCOUT has also introduced InfiniStreamNG(ISNG), a step toward creating a common data platform for both network and security analytics.

NETSCOUT customers have shared with Gartner that the latest version of nGeniusONE has improvedcontextual drill-down for troubleshooting. NETSCOUT customers have also reported limited workflows andtemplate views for IT operators beyond the network domain. Gartner has observed that NETSCOUT customersfrequently have shortlisted Riverbed, CA and ExtraHop before selecting NETSCOUT as their NPMD solution.

These products are well-suited for service providers and large enterprises whose network operations teamsand architects need deep NPMD capabilities at extreme scale to support their own monitoring and diagnosticefforts. Based on Gartner's analysis, the product is best aligned to the network operator role.

NETSCOUT did not respond to requests for supplemental information and/or engage in Gartner's standardprocedures to address the contents of this document. Gartner's analysis for this vendor is based on othercredible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral,public information, and discussions with end users who either have evaluated or deployed each NPMDproduct. NETSCOUT sued Gartner over the content of the 2014 NPMD Magic Quadrant. The court dismissedthe lawsuit, and NETSCOUT has appealed the court's ruling.

Paessler PRTG Network Monitor 17.3Founded in 1997, Paessler is a privately owned company headquartered in Nuremberg, Germany. Paessler'sNPMD product, PRTG Network Monitor v17.3.32, was evaluated for this research. The product is an on-premises software and SaaS offering.

PRTG Network Monitor is a competitively priced consolidated network fault monitoring (NFM), NPMD and ITinfrastructure monitoring (ITIM) tool that relies primarily on SNMP, flow-based and proprietary data gatheringtechniques to present a real-time status of the network. Correlation with fault events and alarm generationagainst service models are standard features. The latest product enhancements include the cloud-basedoffering, including a controller for PRTG in the cloud, to enable starting/stopping of PRTG instances and billingfor the new hosted service. Paessler runs an "anti-aging" program under which it continues to rework itssoftware code. For example, one of the focus areas in the current release was a rework of the architecture tosupport faster development of new feature roll out. Gartner has observed an R&D emphasis over the last 12months with regards to new and improved sensors. For example, the exclusive-use WMI for Windowsmonitoring instead of a hybrid approach, which included Windows performance counters. In November 2017,

Gartner Reprint


the vendor added eight additional NetApp sensors to support the latest and previous versions of NetApp Ontap.The vendor also delivered a REST custom sensor that can query REST APIs.

Gartner's survey of end users rated Paessler strongly in terms of ease of implementation and ease ofupgrading. Paessler's navigation and workflow also scored strongly. The same survey rated Paessler poorly interms of analytics and the ability to commit and deliver on requested enhancements. When asked what othervendors were considered when selecting their NPMD solution, Paessler end users cited SolarWinds mostfrequently, followed by Micro Focus (HPE).

This product is well-suited for SMB, midmarket and managed service providers that need a competitivelypriced NPMD solution with a small deployment footprint and tightly coupled architecture. Based on Gartner'sanalysis, the product is best aligned to the network operator role.

Riverbed SteelCentralFounded in 2002, Riverbed is a privately owned company headquartered in San Francisco, California.Riverbed's NPMD product, SteelCentral, has been evaluated for this research. The product is an on-premisesappliance and software offering.

As one of the handful of NPMD vendors that also have a full-featured application performance monitoringoffering, the solution provides a broad and deep set of application visibility and capabilities. In addition toSteelCentral NPM, the solution consists of multiple components that include NetIM 1.2 (formerly NetSensor3.0.1), UCExpert 7.1.0, Portal 2.0 and Aternity SaaS. The Aternity product has been added (via acquisition) tosupply endpoint monitoring capabilities. SteelCentral's prior NetShark brand has been integrated intoAppResponse and is now only offered as AppResponse.

The solution requires hardware appliances and is sized based on network data storage and processingrequirements. Riverbed has recently introduced a 40G appliance to deal with increasing data rates within thedata center. Riverbed has focused its efforts on integrating Aternity workflows into its NPMD solutions, as wellas into the Portal dashboards. Riverbed has also enhanced data visualization through the use of TruePlotanalytics. Through the relaunch of NetIM for infrastructure monitoring, Riverbed has also sought to improve itscapabilities outside of packet and flow monitoring.

Gartner's survey of end users rated Riverbed highest for contextual drill-down, troubleshooting and diagnostics.Riverbed was rated as having deployments that take longer than most other NPMD vendors, with the majorityof deployments taking three months or longer. When asked what other vendors were considered whenselecting their NPMD solution, Riverbed end users cited Cisco and NETSCOUT most frequently, followed byManageEngine.

This product is well-suited for large enterprises whose NPMD requirements include integration with applicationperformance monitoring (APM) tooling and are intended to support the needs of both network and applicationteams. Based on Gartner's analysis, the product is best aligned to the network operator role.

Savvius Omnipliance, Spotlight, Insight and OmnipeekFounded in 1990, Savvius is a privately held company headquartered in Walnut Creek, California. Savvius'NPMD products (Omnipliance, Omnipeek v10.1.3, Spotlight v11.0.2 and Insight v2.1) were evaluated for thisresearch. These products are on-premises hardware and software offerings.

Savvius' NPMD solution predominantly relies on packet data for network monitoring. Omnipliance is the packet

Gartner Reprint


capture device. Omnipeek provides the network analytics layer, and features include an overview graph with anoverlay summary level information of the entire packet file under analysis. Spotlight is a network monitoringand path quality analysis solution that can provide real-time analysis for up to 20 Gbps. Omnipeek 10 providesthe ability to view file content by reconstructing files through data extraction from reassembled HTTPpayloads. Savvius released Omnipeek v11.1 in November 2017. The new version focuses on improvingcapture-to-disk performance and includes support for NetFlow and IPFIX data sources. VoIP appears to beanother focus of Omnipeek v11.1, with the introduction of playback support for dual-tone multi-frequencing(DTMF) — a new RTP-specific graph to report on jitter and quality for multiple VoIP calls — and new templatereports for VoIP reporting.

Gartner's survey of end users rated Savvius strongly in terms of ease and cost of implementation, and formaintenance and reliability. The same survey rated Savvius poorly in terms of its ability to integrate with third-party tools. When asked what other vendors were considered when selecting their NPMD solution, Savvius endusers cited SolarWinds most frequently, followed by Cisco, NETSCOUT and VIAVI.

This product is well-suited for midmarket and large enterprises seeking a low-cost, packet-oriented NPMD.Based on Gartner's analysis, the product is best aligned to the network operator role.

SevOne Platform and RetinaVuFounded in 2005, SevOne is a privately owned company headquartered in Boston, Massachusetts. SevOne'sNPMD products (SevOne Platform and SevOne RetinaVu 1.0) were evaluated for this research. The productlisting has since been renamed and updated (see below). The vendor is in the process of communicating thetransition of their platform and product names. The offering is composed of on-premises appliance, software,virtual appliance and SaaS products.

SevOne's current offering consists of the SevOne NMS Cluster 5.7, SevOne Data Bus 1.2, SevOne Data Insight1.2, and SevOne User Experience (SUE). SevOne NMS Cluster provides a scalable NPMD solution based ondistributed peer-to-peer architecture, capturing infrastructure metrics, flow records and log data. While NMSCluster lacks native non-flow-based DPI collection and analysis, synthetic analytics and basic packet captureare served through SUE, which is a resell AppNeta Performance Manager. SevOne announced a beta release ofNMS Cluster as a container with the v5.7 release. In September 2017, the vendor announced the SevOne DataPlatform, which is composed of three modules: SevOne Data Engine, SevOne Data Insight and SevOne DataBus. SevOne Data Insight was released in August 2017, acting as a visualization layer, and enables the creationand sharing of real-time reports. SevOne Data Insight is packaged to be delivered in a container as anadditional deployment option. SevOne Data Bus, released with the latest 1.2 version in January 2018, enablescustomers to integrate their network and infrastructure data to external data lakes. SevOne Data Engine will bethe data collection layer and is planned for a 1H18 release. The SevOne Data Platform and its threecomponents supersede the legacy products, namely the SevOne Platform and SevOne RetinaVu. The SevOnePlatform consisted of SevOne Performance Appliance Solution (PAS), Dedicated NetFlow Collector (DNC) andPerformance Log Appliance (PLA).

In Gartner's survey of end users, SevOne was rated strongly in terms of maintenance and reliability, and forcontextual drill-down and troubleshooting. The same survey rated SevOne poorly in terms of ease ofupgrading, the quality of their peer-user community and ease of implementation. When asked what othervendors were considered when selecting their NPMD solution, SevOne end users cited CA, Riverbed andSolarWinds most frequently.

Gartner Reprint


This product is well-suited for large enterprises and service providers that need a flow and log analytics-oriented NPMD solution that can span a distributed and evolving environment. Based on Gartner's analysis, theproduct is best aligned to the IT operations generalist role.

SolarWinds Network Operations Manager 2017.3Founded in 1999, SolarWinds is a privately held company headquartered in Austin, Texas. SolarWinds' NPMDproduct, Network Operations Manager (NOM) 2017.3, was evaluated for this research. This product is an on-premises software offering.

SolarWinds NOM covers monitoring of network elements for fault, availability and performance. NOM is a newproduct that includes the features and functionality of SolarWinds' previous NPMD offerings of NetworkPerformance Monitor (NPM) and NetFlow Traffic Analyzer (NTA), along with additional features. Packetanalysis is included as part of the NPM component but requires the use of separately installed packet capturesensors. NOM also includes some enhancements specific to NOM (for example, a single install and upgradeprocess, and node-based licensing). The intention behind NOM is to provide a simpler evaluation, purchase andusage experience, particularly for larger enterprises. In addition to NOM, SolarWinds introduced PerfStack, adata exploration and correlation interface that utilizes a drag-and-drop interface. Extended support for Merakiand Arista hardware was another notable enhancement.

In Gartner's survey of end users, they rated SolarWinds strongest in terms of the quality of their peer-usercommunity, and among the highest for ease of implementation. The same end users rated SolarWinds amongthe weakest vendors for business transaction monitoring. When asked what other vendors were consideredwhen selecting their NPMD solution, SolarWinds' end users cited Paessler most frequently, followed by Ciscoand ManageEngine.

This product is well-suited for small and midsize enterprises that require an IT infrastructure monitoringsolution with the ability to expand into network traffic analysis. Based on Gartner's analysis, the product is bestaligned to the network architect role.

Statseeker 5.1.2Founded in 1996, Statseeker is a privately held company headquartered in San Diego, California. Stateseeker'sNPMD product, Statseeker v5.1.2, was evaluated for this research. This product is an on-premises softwareappliance offering.

Statseeker comes in lite, classic, professional and enterprise editions. While the evaluated v5.1.2 is an on-premises-only solution, version 5.2 was released on 30 August 2017 and provided a cloud deployment optionon AWS. Statseeker is primarily focused on SNMP, syslog and flow monitoring, while packet-level analysis canbe enabled through a Statseeker module for network segments lacking traditional NetFlow-enabled devices.The UI appears dated but simple, with targeted use cases for arranging performance graphs of multipledevices for side-by-side comparison. Statseeker does not provide application and network dependencymapping in its current version of the product.

Gartner's survey for end users rated Statseeker strongly in terms of ease of implementation, upgrading, and formaintenance and reliability. The same survey rated Statseeker poorly in terms of navigation and workflow, andthe ability to integrate with third-party tools. When asked what other vendors were considered when selectingtheir NPMD solution, Statseeker end users cited Cisco and SolarWinds most frequently.

Gartner Reprint


The product is well-suited for small to midsize organizations that require an IT infrastructure monitoringsolution with the ability to expand into network traffic analysis. Based on Gartner's analysis, the product is bestaligned to the network architect role.

VIAVI Solutions Observer Platform 17.3Founded in 1994, the Enterprise and Cloud Performance Management business unit of VIAVI Solutions isheadquartered in Minneapolis, Minnesota. VIAVI Solutions is a public company. The VIAVI NPMD product,Observer Platform v17.3, was evaluated for this research. The offering is composed of on-premises appliances,software and SaaS products.

The Observer Platform is a packet-oriented NPMD tool that focuses on network and application performancevisibility. The Observer Platform includes the Observer Apex, Observer Analyzer, Observer GigaStor,ObserverLive, Observer Probes, Observer Management Server (OMS) and Observer SightOps. ObserverLIVE isa SaaS-based synthetic monitoring solution suitable for remote monitoring cloud workloads. ObserverSightOps is an OEM product of ScienceLogic that adds active polling of hybrid IT infrastructure.

The latest updates include a "three-steps-to-resolution" workflow that incorporates context sensitivity andautopopulation search functionality. Application dependency visualization has also been improved withsimplified "on-demand" automapping. This provides instant insight into how front-end, middleware and back-end stations are performing.

Gartner's survey of end users indicated the VIAVI NPMD tool was rated strongly in terms of integration withthird-party tools, integration and interoperability, direction and vision, and diagnostics. VIAVI was rated poorlyamong vendors in terms of cost of implementation and cost of maintenance. When asked what other vendorswere considered when selecting their NPMD solution, VIAVI end users cited NETSCOUT and Riverbed mostfrequently, followed by Cisco.

This product is well-suited for large enterprises with deep and varied NPMD requirements to support networkarchitects and network operations specialists. Based on Gartner's analysis, the product is best aligned to thenetwork operator role.

Context

NPMD solutions should be considered as part of an overall network management initiative and included in alarger availability and performance monitoring strategy (which includes AIOps, DEM, APM and ITIMinvestments). Utilizing these additional points of reference will yield further unique criteria (such as existinginvestments, investment plans and vendor relationships) that, when combined with Gartner analysis, can provecritical to proper solution selection.

As part of the Critical Capabilities research process, Gartner conducts a survey of NPMD end users, based onreferences supplied by the vendors listed in this research. This year's survey included 152 respondents fromorganizations across a broad array of industries and company sizes. The results revealed a number of findings:

The end-user survey shows only modest growth in the use of packet analysis, with several major packetmonitoring vendors reporting slowing new customer acquisition. This indicates that the initial interestobserved 12 to 18 months prior to the survey around packet inspection as a data source is on the wane.

Thirty-five percent of surveyed end users of NPMD solutions were outside of I&O, with a large segment

Gartner Reprint


listing themselves as enterprise architects. This underscores a greater trend that NPMD tools are not usedonly by network teams, but are also leveraged across the IT organization, and sometimes even by the line ofbusiness.

Forty-three percent of respondents reported that they will be increasing investment, 50% with no change,and 7% will be decreasing investment.

When asked to name other vendors considered in their NPMD buying decisions, respondents listed thefollowing vendors most often: SolarWinds, NETSCOUT, Cisco and Riverbed.

A number of NPMD vendors have pivoted strongly toward security operations workflows, citing the growingsynergy between NetOps and SecOps (see "Align NetOps and SecOps Tool Objectives With Shared UseCases" ). Thirty-four percent of respondents reported using their NPMD solutions for security analytics, anincrease from 2017.

Product functionality, roadmap and cost were the top three factors driving NPMD buying decisions.

The top three scenarios for use of NPMD solutions were real-time diagnostics, real-time monitoring andalerting, and historical analysis for capacity planning.

Increasing overlaps with IT infrastructure monitoring, APM and AIOps tools are impacting buying decisions.As an example, Splunk was considered as an alternative NPMD vendor for 20% of respondents (see Note 1).

In the course of this research, several additional key trends emerged that should be carefully considered duringNPMD strategy formulation and solution selection, including:

Several large NPMD vendors have undergone reorganizations to align various monitoring assets, often viaacquisition.

We have seen progress in the normalization of UI and workflows across several vendors' toolsets.

End users report that ease of use is still a pain point, and it varies significantly across vendors and withinsolutions.

Support for monitoring cloud and hybrid IT environments remains limited, but this is increasingly appearingon vendor roadmaps for future delivery. IaaS providers, and more specifically Microsoft Azure, haveintroduced their own NPMD solution in the form of Network Watcher, focused exclusively on networkmonitoring within the Azure environment.

Some NPMD vendors have invested in path analytics as part of a cloud monitoring strategy. Some stand-alone vendors have also emerged in this space.

Vendors have increasingly begun to leverage big data repositories and data lakes, or have built operationalanalytics overlays to facilitate data analytics across all captured data.

Most NPMD solutions show only limited support for monitoring SDN, but many NPMD vendors now offervirtual instances of their products to enable deployment within SDN environments. That said, monitoring ofSDN technology remains immature.

NPMD solutions have a primary data source (SNMP, flow or packet), and have a difficult time leveraging thebenefits of other data sources for the maximum value to the user.

Gartner Reprint


Many NPMD solutions are assembled from multiple products. This can enable modular adoption of NPMDcapabilities, but it can also add significant complexity to procurement and ongoing maintenance. It is oftenmore difficult to integrate modules than customers expect or as vendors frequently claim.

Organizations must purchase tools that closely match their current maturity levels (see "ITScore forInfrastructure and Operations" ). Many network monitoring teams have yet to make a successful leap frombasic, reactive network availability management to proactive performance management. NPMD and analyticswill also be key elements to the success of any IBNS initiative, which itself is a path toward furtherorganizational maturity. While tool investment can play a part in this maturation, simply investing in NPMDtooling without similar investments in training, integration and processes will yield limited results, at best. Toprovide this perspective, IT operations teams must assess the current state of maturity regularly and set clear,measurable objectives both individually and at the organizational level.

I&O leaders should build a list of criteria describing their current and future needs, and then select from vendorsthat best meet those requirements. They should select a vendor that has both a history of, and future plans for,focusing on this market. Give careful consideration to required skills, training, processes and deploymentinvestments. These factors will have a much greater impact on the overall value realized from an NPMDinvestment than any specific functional capability found in a given tool.

Product/Service Class Definition

NPMD tools typically include the following primary capabilities:

Passive monitoring techniques that analyze network traffic seen by network probes or appliances attachedto mirror ports on network switches, or summarized from flow-based instrumentation embedded in networkhardware, to provide a high-level summary of network conversations.

The ability to inspect the contents of the network packet in the context of voice-, video- and network-basedapplications. This is essential for the forensics of network-based issues. This product must have the ability todecode and present data on the application, not just expose the raw packet data.

The ability to create, manage and report on synthetic test transactions, which leverage features normallyembedded in the network hardware device's OS, such as Service Assurance Agent (SSA), IP SLAs andMediaNet.

The ability to generate high-level dashboards and metric roll-ups around overall network performance andhealth.

A set of predefined, out-of-the-box and static thresholds indicating warning and critical conditions forrelevant metrics that are monitored, and escalation of threshold violations to event correlation and analytics(ECA) and notification management systems. These thresholds must be user-configurable.

The approach most enterprise and public-sector organizations have taken in monitoring the network has beenfocused on network probes and via SNMP polling of network devices. SNMP polling provides valuable volume,error and utilization data, but lacks granularity by not breaking down the usage of the bandwidth (see Note 2).Probe packet-based inspection was introduced to fill this gap, but due to the cost of devices, as well asmanagement and implementation complexity, probes cannot provide visibility in all physical and logicalnetwork locations (see Note 3). This requirement for better data than SNMP polling, at a level that is more cost-effective than deploying a large number of packet inspection probes, resulted in the emergence of summarized

Gartner Reprint


flow-based techniques (see Note 4).

Understanding how tool functionality has evolved for each data source, along with the caveats at each level, iscrucial when determining which NPMD tool is appropriate for your organization.

This research examines five critical capabilities that differentiate NPMD products in various use cases:

Endpoint/component/link monitoring

Service delivery monitoring

Diagnostics

Analytics

Integration and interoperability

These critical capabilities represent the most important and differentiated features and functions for NPMDtools. The capabilities in each of these areas were weighted for their respective use cases.

Critical Capabilities Definition

Endpoint/Component/Link MonitoringThe ability to monitor, diagnose and generate alerts for endpoints, network components and network links.

Endpoints can include (but at not restricted to) servers, virtual machines (VMs), storage systems or anythingwith an IP address. This includes cloud-hosted and wireless endpoints. Network components can includerouters, switches and other network devices. Network links refer to the connectivity between network-attachedinfrastructure.

Service Delivery MonitoringService delivery dashboards are key to all use cases because they help IT organizations understand alignmentwith IT objectives, as shown by key performance indicators (KPIs). Reporting is evaluated on the product'sability to provide trend reports that measure productivity across core processes.

Dashboard and reporting capabilities must be tied to business value metrics, common critical success factorsand KPIs. The tool must offer multidimensional charts that show how related metrics affect each other. Thesewould be based on the tool's ability to monitor, diagnose and generate alerts for dynamic end-to-end networkservice delivery as it relates to the end-user experience, business service delivery and infrastructure componentinteraction.

DiagnosticsThe ability to facilitate or automate typical steps taken by an operator to troubleshoot, triage and diagnoseissues. Aspects that enhance a tool's diagnostics include contextual drill-down, intuitive workflow, navigationalaids (such as hover-over features) and simple search mechanisms.

AnalyticsThe ability to leverage one or more algorithmic techniques to distill, infer and/or create action-oriented advicefrom the dataset collected, which can include metric, time-series-based and unstructured data sources.

Algorithmic capabilities include the ability to support model discovery, ascertain the root causes of

Gartner Reprint


performance problems, assess the impacts of multiple known root causes, predict events and recommendremediation. Model discovery is the basis of the other four use cases. Many NPMD tools claiming algorithmiccapabilities often only filter, aggregate and visualize data. Such tools do not perform inferences. At best, theyonly facilitate inferences taking place within a human mind (diagnostics). It is important to understand thedistinction, regardless of whether automated model discovery, root cause analysis, cause impactdetermination, prediction or action recommendation is the desired requirement.

Integration and InteroperabilityThe tool's ability to be integrated with other third-party tools, and the ability of those third-party tools tointegrate with NPMD tools, are increasingly important, particularly for organizations that license best-of-breedITOM products, as opposed to products from a single vendor.

Products are evaluated on the presence of out-of-the-box integration capabilities (via APIs or connectors) thatlink to NFM, network automation, ECA tools, APM, ITIM and unified communications monitoring (UCM) tools.

Use Cases

Network OperatorResponsibilities include troubleshooting network-related hardware/software, monitoring and configuringnetwork devices, and making recommendations to network architects.

Network operators are typically members of a team responsible for monitoring and maintaining the networkinfrastructure. Extended responsibilities may also include the analysis of network components for correctconfiguration (including patching), upgrades and making recommendations for network enhancements. Inenterprises and government organizations, they may also serve as liaisons with communications serviceproviders to ensure the proper service delivery, operation and compliance of the provider's equipment, as wellas to ascertain and delineate responsibility when fault isolation is not immediately apparent.

Network ArchitectResponsibilities include conceptual network design, planning, governance, communicating architecturerationale to technical and business stakeholders, and escalation handling.

Unlike generalists, engineers and operators, architects are largely proactive and work predominantly onplanning, documentation and optimization, new projects, and infrastructure strategy. Network architects oftenreport outside the I&O teams, typically working in an enterprise architecture or project management office.Network architects are relied on to provide functional, technical and/or process leadership. Common extendedresponsibilities include the formulation of best practices to ensure effective workflow, and the monitoring ofindustry/technological trends, adoption and direction.

IT Operations GeneralistResponsibilities include end-user support, fault validation, service desk ticket generation, troubleshooting broadIT issues and working with multiple domain specialists.

IT operations generalists are typically members of a team that is responsible for the monitoring and support ofthe IT infrastructure. This can include endpoints, servers, applications and/or the network (including voice andvideo services). Given their broad jurisdiction, IT operations generalists are normally interfacing with fault andavailability monitoring tools and, in less mature organizations, they deal with issues reactively, as they are

Gartner Reprint


being reported by end users. Collaboration with server, database, application support, application development,network, storage and security teams is common. As a result, fault identification and isolation is a frequent task,as is problem handoff.

IT Operations ManagerResponsibilities include team management, service compliance, escalation handling, cross-domain experience,report generation and continuous service improvement.

IT operations managers typically oversee a team of IT operations generalists. As a result, they are often a pointof escalation and require adequate high-level visibility as to the immediate health of business applications andservices, as well as the underlying infrastructure that supports them. Historical and performance reportingassists IT operations managers in moving to more proactive states, whereby emerging issues can beanticipated and dealt with prior to end users being affected. Tracking and reporting of SLA compliance, as wellas identifying cross-domain constraints, are also requirements that an IT operations manager would seek fromany tool.

Line-of-Business UserResponsibilities include planning and reporting divisional and regional activities, and performance in the serviceof business objectives (for example, growth, cost and risk).

The line of business (LOB) user may be a manager or member of a team that is either wholly or partiallyresponsible for those applications or services that are perceived to be vital to running the enterprise orbusiness unit. A LOB user will have a requirement to collect, identify and analyze business data to review andevaluate the organization's business performance. Identifying constraints to the delivery of the service, as wellas impediments to maximizing its efficiency and performance, will be priorities. Adherence to regulatorycompliance while reporting on KPI trends and patterns is also vital. Integration with business reporting andOLAP tools will be a requirement for LOB users.

Vendors Added and Dropped

AddedColasoft

H3C

ManageEngine

Micro Focus (HPE)

Performance Vision

Savvius

Statseeker

DroppedHPE — The NPMD suite formerly associated with Hewlett Packard Enterprise (HPE) was evaluated under the"Micro Focus (HPE)" label following the acquisition of the software assets by the U.K.-based firm.

Gartner Reprint


Genie Networks — Focus is on security workflows and use cases.

Inclusion Criteria

Product-Related Criteria

Vendors are required to meet the following criteria to be considered for the 2018 NPMD Magic Quadrant andCritical Capabilities research:

The ability to monitor, diagnose and generate alerts for:

Network endpoints — Servers, virtual machines, storage systems or anything with an IP address bymeasuring these components directly, in combination with a network perspective. This includes cloud-hosted and wireless endpoints.

Network components — Routers, switches and other network devices. This includes SDN and NFVcomponents.

Network links — Connectivity between network-attached infrastructure.

The ability to monitor, diagnose and generate alerts for dynamic end-to-end network service delivery, as itrelates to:

End-user experience — The capture of data about how end-to-end application availability, latency andquality appear to the end user from a network perspective. This is limited to network traffic visibility and isnot within components, such as what application performance monitoring is able to accomplish.

Business service delivery — The speed and overall quality of network service and/or application delivery tothe user in support of key business activities, as defined by the operator of the NPMD product. Thesedefinitions may overlap as services and applications are recombined into new applications.

Infrastructure component interactions — The focus on infrastructure components as they interact via thenetwork, as well as the network delivery of services or applications.

Support for analysis of:

Real-time performance and behaviors — Essential for troubleshooting in the current state of theenvironment. Analysis of data must be done within three minutes under normal network loads andconditions.

Historical performance and behaviors — To help understand what occurred or what is trending over time.

Predictive behaviors by leveraging analytics technologies — The ability to distill and create actionableadvice from the large dataset collected across the various data sources.

The ability to leverage the following data sources:

Network-device-generated traffic data, including flow-based data sources inclusive of NetFlow and IPFIX.

Network-device-generated health data.

Network packet analysis to identify application types and performance characteristics.

Gartner Reprint


The ability to support the following scalability and performance requirements:

Real-time monitoring of 10G Ethernet networks at full line rate.

Ingesting of sampled flow records at a rate of 75,000 flows per second via a single instance of theproduct.

Non-Product-Related Criteria

Total NPMD product revenue (including new licenses, updates, maintenance, subscriptions, SaaS, hostingand technical support) must have exceeded $7.5 million for 3Q16 through 2Q17, excluding revenue derivedfrom security-related buying centers.

The vendor must have at least 75 customers that use its NPMD product actively in a productionenvironment.

The vendor must have at least 10 customers located in at least two of the following geographic locations:North America, Latin America, EMEA and/or the APAC region that use its NPMD product actively in aproduction environment.

Table 1. Weighting for Critical Capabilities in Use Cases

Critical Capabilities

Endpoint/Component/Link Monitoring

Network Operator 23%

Network Architect 32%

IT Operations Generalist 10%

IT Operations Manager 5%

Line-of-Business User 0%

Service Delivery Monitoring





Gartner Reprint



Diagnostics






Analytics






Integration and Interoperability

Network Operator 5%





Total

Gartner Reprint








This methodology requires analysts to identify the critical capabilities for a class of products/services. Eachcapability is then weighed in terms of its relative importance for specific product/service use cases.

Critical Capabilities Rating

Each of the products/services has been evaluated on the critical capabilities on a scale of 1 to 5; a score of 1 =Poor (most or all defined requirements are not achieved), while 5 = Outstanding (significantly exceedsrequirements).

Table 2. Product/Service Rating on Critical Capabilities

Critical Capabilities

Endpoint/Component/Link Monitoring

Accedian (Performance Vision) Performance Vision 4.2 2.0

AppNeta Performance Manager 10.8 3.2

CA Technologies Network Operations and Analytics 3.3

Cisco Prime Infrastructure, NAM and Tetration Analytics 3.0

Colasoft UPM, nChronos and Capsa 3.3

Corvil Analytics 9.2 2.3

ExtraHop Platform 7.0 2.5

Flowmon Networks 9.0 3.0

Gartner Reprint


H3C iMC 7.3 3.0

InfoVista VistaInsight for Networks, VistaGo and 5View Suite 3.1

Ipswitch WhatsUp Gold 2017 Plus 2.0

LiveAction LiveNX 6.2 3.1

ManageEngine OpManager 12.3 2.3

Micro Focus NNMi and RUM 2.9

NETSCOUT nGeniusONE and nGeniusPULSE 4.3

Paessler PRTG Network Monitor 17.3 2.5

Riverbed SteelCentral 4.3

Savvius Omnipliance, Spotlight, Insight and Omnipeek 2.0

SevOne Platform and RetinaVu 2.8

SolarWinds Network Operations Manager 2017.3 3.2

Statseeker 5.1.2 2.5

VIAVI Solutions Observer Platform 17.3 4.2

Service Delivery Monitoring





Gartner Reprint






H3C iMC 7.3 3.0














Diagnostics


Gartner Reprint









H3C iMC 7.3 3.0













Gartner Reprint



Analytics









H3C iMC 7.3 2.0










Gartner Reprint






Integration and Interoperability









H3C iMC 7.3 1.0






Gartner Reprint











Table 3 shows the product/service scores for each use case. The scores, which are generated by multiplyingthe use-case weightings by the product/service ratings, summarize how well the critical capabilities are met foreach use case.

Table 3. Product Score in Use Cases

Use Cases

Network Operator








Gartner Reprint



H3C iMC 7.3 2.73














Network Architect





Gartner Reprint






H3C iMC 7.3 2.53














IT Operations Generalist


Gartner Reprint









H3C iMC 7.3 2.27












Gartner Reprint




IT Operations Manager









H3C iMC 7.3 2.20









Gartner Reprint







Line-of-Business User









H3C iMC 7.3 2.45






Gartner Reprint











To determine an overall score for each product/service in the use cases, multiply the ratings in Table 2 by theweightings shown in Table 1.

Acronym Key and Glossary Terms

AIOps

API

Artificial Intelligence for IT Operations Application Programming Interface

BGP

Artificial Intelligence for IT Operations Border Gateway Protocol

CSP

Artificial Intelligence for IT Operations Communications Service Provider

DEM

Artificial Intelligence for IT Operations Digital Experience Monitoring

IBNS

Artificial Intelligence for IT Operations Intent-Based Networking Systems

Gartner Reprint


IP SLA

Artificial Intelligence for IT Operations Internet Protocol Service-Level Agreement

MPLS

Artificial Intelligence for IT Operations Multiprotocol Label Switching

NFV

Artificial Intelligence for IT Operations Network Function Virtualization

OLAP

Artificial Intelligence for IT Operations Online Analytical Processing

Evidence

Observations are based on the Magic Quadrant and Critical Capabilities reference survey and over 600 Gartnerclient inquiries and engagements with end users and vendors over the last 12 months that have reported:

Increased demand for network visibility from other aspects of the business

Initial immaturity of managing network environments that NPMD tools help address

Extended requirements for NPMD tools to help justify network investments and aid capacity planning

The need for tighter SLA compliance, especially in light of increased adoption of IP-based voice, video andUC services, and hosted virtual desktop (HVD) services

The following press releases were also used:

"SolarWinds Acquires AppNeta TraceView Solution," (SolarWinds)

"NETSCOUT Unveils Industry's First Real-Time Information Platform for Service Assurance, Cybersecurity and BigData," (NETSCOUT)

"Micro Focus Announces Intent to Merge With Hewlett Packard Enterprise's Software Business Segment," (MicroFocus [HPE])

Note 1 Honorable Mentions

The following sample vendors were unable to meet the inclusion criteria to be included in this research, but docome up in NPMD-related inquiries:

Catchpoint

Dynatrace

Empirix

https://www.solarwinds.com/company/press-releases/solarwinds-acquires-appneta-traceview-solution

https://www.netscout.com/news/press-release/real-time-information-platform

https://www.netscout.com/news/press-release/real-time-information-platform

https://www.microfocus.com/about/press-room/article/2016/micro-focus-announces-intent-to-merge-with-hewlett-packard-enterprise-software-business-segment/

Gartner Reprint


FixStream

Kentik

Microsoft Azure

Netis Systems

Nyansa

Splunk

ThousandEyes

Note 2 SNMP as an NPMD Data Source, and SNMP Polling

Period polling is one method that looks to quantify network usage of network elements to gauge therequirements of the infrastructure.

A polling-based approach can assess the existing environment and poll via standard SNMP managementinformation bases (MIBs) to trend utilization on each network element. SNMP polling can also be used togather error and volume data (for instance, triggering an alert when a fault occurs or a predetermined thresholdis breached). For example, it is possible to monitor the CPU, voltage and temperature of a device. If any ofthese parameters increase above expected levels, this may require investigation to prevent failure. In additionto error and volume monitoring, SNMP polling can be used to collect basic performance metrics to ascertainhow much traffic is flowing through a network device.

Based on the metrics gathered, the network team can estimate the delta between existing and requiredbandwidth needs on a per-location basis. A limitation with this method is the level of granularity that thisapproach offers. Data traffic is "bursty" in nature, which may be masked by polling intervals. The lack of"application-aware" segregation prevents the correlation of performance to specific applications.

Take the following example: Location A has a constant 50% utilization over the 10-minute polling period.Location B has utilization of 100% in the first five minutes of the period and 0% in the second five minutes. Apolling analysis would indicate the two locations have a 50% average. The skewed data presents a misleadingpicture. While Location A has sufficient contingency in terms of bandwidth, Location B is at full capacity for halfof the time. If new bandwidth-intensive applications were deployed at Location B, this will likely severely impactall services running at this site.

An increase of polling frequency is an option. However, this adds to the overall overhead of the network withevery additional request for data and the response given from each element. Balancing the need for granularitywith oversaturation of the network with management traffic is a fine line.

Note 3 Packet-Based Technology as an NPMD Data Source

Another approach is to examine the infrastructure in detail on a per-packet basis as opposed to samplingduring a one- to 15-minute time frame. Instead of receiving a single data point every 10 minutes as per the

Gartner Reprint


example in Note 2, there would be data points for every packet traversed in the same 10-minute period, givingvisibility into volatile traffic behavior from bursty application types.

If raw packet data is being gathered and examined, then the information collected is vendor-independent.Hence, the data has not been interpreted, summarized or manipulated in any fashion whatsoever. As a result,those relying on this information will have increased confidence in the accuracy of the data on which they aremaking decisions.

This approach affords far greater insight. However, the offset is that a physical appliance or "probe" must besituated at each location. Otherwise, such detailed information cannot be gathered. These appliances have acost implication that can be excessive for an enterprise organization. Finally, encrypted traffic can inhibitmaximum insight from this data source.

Note 4 Flow-Based Technology as an NPMD Data Source

A final method to provide further granularity is to take advantage of flow-based summarized data that providesthe conversation between two network nodes (known as flow). Flow records also consume comparativelyminimal storage space, compared with the raw packets that probe-based solutions leverage and may store.For example, as organizations move to 100GB networks, packet collection techniques become challenging,whereas flow-based methods remain a more practical option. Flow-based data does not provide the detaildown to a specific set of network packets going between the source and the destination, and can have animpact on the devices where it is implemented.

There are several flow collection standards. Vendor-derived standards are predominant and may notnecessarily expose weaknesses to which the vendor may not wish to draw attention. However, they are well-integrated with the devices they are designed to monitor and are updated in anticipation of new hardwarereleases. The emergence of the latter two standards (IPFIX and sFlow) is positive because they are vendor-agnostic and, therefore, facilitate the management of a heterogeneous environment in a consistent manner.

Comparing different vendors' devices is also facilitated. Such flow collection technology standards provide thesame broad feature sets. Flow data collection is embedded in network devices themselves. The deviceanalyzes the network traffic traversing from one interface to another with the primary purpose of assessingbandwidth consumption and the level of data being sent and received between various source and destinationports across the network. That data is then summarized into a stream of flow records that are sent to themonitoring tools collecting and assessing the flow records.

Flow collection capabilities provide insight into which devices and applications are consuming bandwidth, howlong the conversations are lasting, and who is having these conversations. Since the data is summarized, adegree of detail is removed in order to simplify the ability to process and extract meaning from the actualnetwork data.

Critical Capabilities Methodology

This methodology requires analysts to identify the critical capabilities for a class of products or services. Eachcapability is then weighted in terms of its relative importance for specific product or service use cases. Next,products/services are rated in terms of how well they achieve each of the critical capabilities. A score that

Gartner Reprint


summarizes how well they meet the critical capabilities for each use case is then calculated for eachproduct/service.

"Critical capabilities" are attributes that differentiate products/services in a class in terms of their quality andperformance. Gartner recommends that users consider the set of critical capabilities as some of the mostimportant criteria for acquisition decisions.

In defining the product/service category for evaluation, the analyst first identifies the leading uses for theproducts/services in this market. What needs are end-users looking to fulfill, when consideringproducts/services in this market? Use cases should match common client deployment scenarios. Thesedistinct client scenarios define the Use Cases.

The analyst then identifies the critical capabilities. These capabilities are generalized groups of featurescommonly required by this class of products/services. Each capability is assigned a level of importance infulfilling that particular need; some sets of features are more important than others, depending on the use casebeing evaluated.

Each vendor’s product or service is evaluated in terms of how well it delivers each capability, on a five-pointscale. These ratings are displayed side-by-side for all vendors, allowing easy comparisons between thedifferent sets of features.

Ratings and summary scores range from 1.0 to 5.0:

1 = Poor or Absent: most or all defined requirements for a capability are not achieved

2 = Fair: some requirements are not achieved

3 = Good: meets requirements

4 = Excellent: meets or exceeds some requirements

5 = Outstanding: significantly exceeds requirements

To determine an overall score for each product in the use cases, the product ratings are multiplied by theweightings to come up with the product score in use cases.

The critical capabilities Gartner has selected do not represent all capabilities for any product; therefore, may notrepresent those most important for a specific use situation or business objective. Clients should use a criticalcapabilities analysis as one of several sources of input about a product before making a product/servicedecision.

© 2018 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. or its affiliates.This publication may not be reproduced or distributed in any form without Gartner's prior written permission. If you areauthorized to access this publication, your use of it is subject to the Usage Guidelines for Gartner Services posted ongartner.com. The information contained in this publication has been obtained from sources believed to be reliable. Gartnerdisclaims all warranties as to the accuracy, completeness or adequacy of such information and shall have no liability for errors,omissions or inadequacies in such information. This publication consists of the opinions of Gartner's research organizationand should not be construed as statements of fact. The opinions expressed herein are subject to change without notice.Gartner provides information technology research and advisory services to a wide range of technology consumers,manufacturers and sellers, and may have client relationships with, and derive revenues from, companies discussed herein.

https://www.gartner.com/technology/contact/become-a-client.jsp?cm_sp=bac-_-reprint-_-banner

Gartner Reprint


Although Gartner research may include a discussion of related legal issues, Gartner does not provide legal advice or servicesand its research should not be construed or used as such. Gartner is a public company, and its shareholders may include firmsand funds that have financial interests in entities covered in Gartner research. Gartner's Board of Directors may include seniormanagers of these firms or funds. Gartner research is produced independently by its research organization without input orinfluence from these firms, funds or their managers. For further information on the independence and integrity of Gartnerresearch, see "Guiding Principles on Independence and Objectivity."

About | Careers | Newsroom | Policies | Privacy | Site Index | IT Glossary | Contact Gartner

http://www.gartner.com/technology/about.jsp

http://www.gartner.com/technology/careers/

http://www.gartner.com/newsroom/

http://www.gartner.com/technology/about/policies/guidelines_ov.jsp

https://www.gartner.com/privacy

http://www.gartner.com/technology/site-index.jsp

http://www.gartner.com/it-glossary/

http://www.gartner.com/technology/contact/contact_gartner.jsp

Critical Capabilities for Network Performance …...improved network capacity management, thereby...

Documents

Transcript of Critical Capabilities for Network Performance …...improved network capacity management, thereby...