Comparative Analysis of IT Monitoring Tools

Comparative Analysis of IT Monitoring Tools Review of IT monitoring platforms from CA Technologies, SolarWinds, IBM, and Nagios The following analysis compares strengths and weaknesses of CA Nimsoft Monitor version 7.1 and the new CA Nimsoft Monitor version 7.5 to trial IT monitoring tools versions from IBM, SolarWinds, and Nagios. Apprize360 Intelligence, INC. June 26, 2014 A Special Report Commissioned for CA Technologies, Inc.

2 | P a g e

Table of Contents

Overview and Summary ..................................................................................................................... 3

Study Methodology ........................................................................................................................... 3

Enhancements from Nimsoft Monitor Version 7.1 to 7.5 ..................................................................... 4

CA Nimsoft Monitor V7.5 Configuration and Installation Process ......................................................... 5

Comparative Benchmark Results ........................................................................................................ 6

CA Nimsoft Monitor vs. SolarWinds .......................................................................................................................... 7

CA Nimsoft Monitor vs. Nagios XI ............................................................................................................................. 7

CA Nimsoft Monitor vs. IBM SmartCloud Monitoring ............................................................................................... 8

Remote Monitoring Considerations for Distributed Environments ....................................................... 9

Assessment of Specific Testing Scenarios .......................................................................................... 11

Monitoring Configuration ........................................................................................................................................ 12

Out-of-the-box Unified Dashboards ........................................................................................................................ 13

SLA Reporting .......................................................................................................................................................... 15

Ease of Use for Alarms, Dashboards, & Reporting .................................................................................................. 16

Summary and Conclusions ................................................................................................................ 18

Figure 4: Total Time-to-Monitor .............................................................................................................................. 18

Table 4: Comparative Summary - Total Time-to-Monitor ....................................................................................... 19

Appendix A: Tables .......................................................................................................................... 20

Table 5: CA vs. SolarWinds ...................................................................................................................................... 20

Table 6: CA vs. IBM & Nagios .................................................................................................................................. 22

Table 7: Response to Testing Scenarios ................................................................................................................... 24

Product names mentioned herein may be trademarks and/or registered trademarks of their respective companies.

The information contained in this publication has been obtained from sources believed to be reliable. Apprize360, LLC disclaims all warranties as to the accuracy, completeness, or adequacy of such information and shall have no liability for errors, omissions, or inadequacies in such

information. This publication consists of the opinions of Apprize360’s research organization and should not be construed as presenting statements of fact. The opinions expressed herein are subject to change without notice.

3 | P a g e

Overview and Summary

CA Nimsoft Monitor is a unified IT monitoring and

management solution that offers deep monitoring

functionality within a single, unified architecture. CA

Technologies recently launched Nimsoft Monitor

version 7.5, bringing new functionality to the platform,

as well as improvements in overall usability, navigation,

and reporting.

Multiple IT monitoring platforms are available on the

market today, supporting the various needs of small,

medium-sized, and large enterprises, as well as

managed service providers (MSPs). To better

understand how CA Nimsoft Monitor compares to other

IT monitoring platforms in terms of functionality,

operations, and usability, Apprize360 Intelligence

studied and compared eight different IT monitoring

products including:

1. CA Nimsoft Monitor Version 7.1

2. CA Nimsoft Monitor Version 7.5

3. SolarWinds Network Performance Monitor

(NPM)

4. SolarWinds Server and Application Monitor

(SAM)

5. SolarWinds Virtualization Manager

6. SolarWinds Storage Manager

7. Nagios XI

8. IBM SmartCloud Monitoring

Study Methodology

Apprize360 downloaded fully functional trial versions of each of the eight IT monitoring products

included in this study, so as to understand the deployment and operational experience that a

typical customer would encounter with each platform. The comparative analysis was performed

using a Dell PowerEdge T620 Server with 768GB of memory across 24 DIMM slots and 64TB of

disk space. For CA Nimsoft Monitor, IBM SmartCloud Monitoring, and the four SolarWinds

products, the operating system used was Microsoft Server 2012. For Nagios XI, the same server

Key Study Findings

CA Nimsoft Monitor version 7.5 is a

single, unified solution, and not several

point products integrated together with

a common GUI. This significantly reduces

the time IT staff spends in trouble-

shooting issues, generating day-to-day

reports and managing their monitoring

tools.

CA Nimsoft Monitor 7.5 was the easiest

platform to download and begin using;

embedded interactive guides allowed for

faster implementation due to reduced

initial configuration errors.

CA Nimsoft had the “fastest time to

monitor” (time from download to active

device monitoring) among the products

studied, providing a more rapid time to

value.

CA Nimsoft Monitor supports remote

monitoring needs without requiring a

dedicated VPN connection to remote

sites.

4 | P a g e

was used, but it was partitioned to run Linux Red Hat Enterprise. In all cases, configuration and

discovery was completed on a network with 100 end devices. Downloads of the trial versions of

the platforms were carried out using a T1 connection that averaged 20-30MBPS Internet speeds.

Enhancements from Nimsoft Monitor Version 7.1 to 7.5

CA Technologies launched its latest version of Nimsoft Monitor on April 2, 2014 called CA Nimsoft

Monitor Version 7.5. Significant improvements were made from CA Nimsoft Monitor Version 7.1

to 7.5, primarily in the following five areas:

1. Faster and easier platform deployment

2. New and improved maintenance mode

3. HTML5 access and improved custom dashboards

4. Improved dynamic thresholds

5. Enhanced discovery capabilities

Faster and Easier Platform Deployment

CA Technologies has invested significantly in the deployment experience of CA Nimsoft Monitor

version 7.5. The platform uses a self-extraction application from Flexera Software’s

InstallAnywhere, making the guided downloaded experience of version 7.5 easy and quick. CA

has included helpful on-screen tips and installation guide points, which allow IT administrators

to prepare for future configuration and installation steps with CA Nimsoft Monitor.

New and Improved Maintenance Mode

CA Technologies has significantly enhanced CA Nimsoft Monitor’s maintenance mode capabilities

in version 7.5. The maintenance mode probe allows users to bring systems in and out of

maintenance mode with a push of a button. When systems are in maintenance mode, alarms for

version 7.5 are suppressed, so that network changes can be made without alarm generation. New

functionality also supports alarm suppression during regular or scheduled network maintenance,

in order to avoid false alarms and alerts.

HTML5 Access and Improved Custom Dashboards

Mobile access and custom reporting are two additional areas that CA Technologies has

strategically invested in for CA Nimsoft Monitor version 7.5. The CA Nimsoft Monitor dashboard

portlet in the Unified Management Portal (UMP) has been improved for added usability,

reliability, and performance. In version 7.5, users can also access CA Nimsoft Monitor dashboards

on mobile devices. The CA Nimsoft Monitor UMP Dashboard portlet replaces the Custom

Dashboards, Dashboard Designer, and Dynamic Views portlets in previous versions.

5 | P a g e

Improved Dynamic Thresholds

In CA Nimsoft Monitor version 7.5, the baseline engine probe allows users to set dynamic

thresholds for any probe that has alarm thresholds. CA Technologies has also improved the

overall performance and stability of hub-to-hub tunnels. Additionally, the CA Nimsoft Monitor

RESTful Web Services API has been enhanced, through improvements including:

Added REST API for Maintenance Mode

Added call to REST API QoS data retrieval, to retrieve QoS Constraint IDs for an SLO

Added REST API for enriched origins

Enhanced Discovery Capabilities

In CA Nimsoft Monitor version 7.5, the discovery process has been improved to include:

Improved SNMP characterization

Improved LUA functionality for Discovery Server

Improved NDP discovery for Discovery Agent

Configurable SNMP port

CA Nimsoft Monitor V7.5 Configuration and Installation Process

The installation process for CA Nimsoft Monitor version 7.5 is fundamentally different from those

of the other IT monitoring products offered by IBM, Nagios, and SolarWinds. CA Technologies has

made sure that everything – from the product download to the installation process – is seamless

and easy to complete. CA Technologies uses Flexera Software’s InstallAnywhere platform for CA

Nimsoft Monitor installation, which makes installation and execution quick and uncomplicated.

The only prerequisite is that the host server must be running Java in order for the installation to

be completed.

During the installation process, CA Nimsoft Monitor displays on-screen hints to prepare the IT

administrator for the subsequent configuration and device discovery phases. Both Nagios and

SolarWinds utilize a flat transfer of a ZIP file, which requires separate execution and manual file

extraction. While no download or installation errors occurred with any of the platforms reviewed,

the download and installation process of CA Nimsoft Monitor version 7.5 was by far the easiest

to use of all of the products assessed during this study.

After the initial installation, the configuration of CA Nimsoft Monitor version 7.5 continues to be

straightforward. Of all of the vendor platforms reviewed, CA Nimsoft Monitor version 7.5 was by

far the easiest and quickest to configure and begin monitoring. The out-of-the-box automated

discovery process took 15 minutes to identify and log all 100 devices. After the automated

6 | P a g e

discovery process was completed, CA Nimsoft Monitor immediately began monitoring devices

for performance, highlighting any network latency and individual device performance issues.

The CA Nimsoft Monitor Unified

Service Manager (USM)

dashboard gives the user a

complete view of the entire IT

environment being monitored.

This includes supporting

dynamic drill-down into specific

areas of concern, right from the

USM dashboard. Within the

dashboard, users can also

identify device performance issues by geography, measure overall performance, and review the

latest alarms. The single dashboard provides a complete, unified view of an IT infrastructure, and

is easy to access throughout the product, whenever the user wants to proactively review

performance. However, if a user desires a more reactive approach, CA Nimsoft Monitor alarm

capabilities allow for specific thresholds to be established for device and network performance.

CA Nimsoft Monitor will then alert the IT administrator through an alarm or email notification

concerning any performance-related alarms. This is another area in which version 7.5 has been

enhanced in comparison with version 7.1. The dashboard portlet in USM provides improved

usability, reliability, and performance, and allows the user to view dashboards on mobile tablet

devices.

Reporting is a significant strength of CA Nimsoft Monitor version 7.5. The reporting process is

completely interactive, allowing the user to drag-and-drop specific devices onto the reporting

screen for immediate graphing. Reporting can be initiated by a specific device, by a group of

devices, or by quality of service. The platform’s advanced filtering capabilities enable granular

reporting down to the specific probe level. In addition, Nimsoft version 7.5 supports the ability

to import non-IT data, such as sales and revenue, to display alongside IT data.

Comparative Benchmark Results

Apprize360’s assessment discovered several differences in the installation and configuration

processes, as well as in overall operations, between the various platforms included in this study.

Overall, CA Nimsoft Monitor version 7.5 led the group, with regard to overall ease-of-installation,

configuration, and operations, as well as speed of download and installation. In addition, unlike

SolarWinds and Nagios, CA Nimsoft Monitor offers more than a time-limited trial. SolarWinds

only allows users 30 days of free use of its respective platforms, while Nagios provides a 60-day

trial period. By comparison, CA offers a free version of the product, called CA Nimsoft Monitor

CA Nimsoft Monitor Version 7.5 Unified Dashboard

7 | P a g e

Snap, which allows users to monitor up to 30 devices for as long as they want. For more info and

to download, visit: www.ca.com/snap.

The full quantitative results of this study can be found in Appendix A of this document, beginning

on page 20.

CA Nimsoft Monitor vs. SolarWinds

All four of the assessed SolarWinds products use the same download and installation method.

SolarWinds requires users to download a ZIP file containing an auto-extractable application that

must then be manually activated. Unlike CA Nimsoft Monitor, SolarWinds offers no guided

installation wizard to help the user through the installation process. In addition, the SolarWinds

products involve several different steps that are needed to complete the installation. It took 61

minutes to download, install, configure, and discover 100 devices with CA Nimsoft Monitor.

However, more than double that time was required to complete the same installation process

with the four SolarWinds products. SolarWinds Virtualization Manager and SolarWinds NPM both

took the longest to deploy, requiring roughly two hours before active monitoring could begin.

Table 1: CA Nimsoft Monitor 7.5 vs. SolarWinds

CA Nimsoft Monitor 7.5

SolarWinds NPM

SolarWinds Storage Mgr

SolarWinds Virtualization

Mgr

SolarWinds SAM

Download 15 mins 11 mins 8 mins 15 mins 10 mins

Installation 15 mins 52 mins 44 mins 73 mins 60 mins

Configuration 16 mins 25 mins 38 mins 44 mins 33 mins

Discovery 15 mins 25 mins 35 mins 32 mins 35 mins

Total Time to Monitor

61 minutes 113 minutes 125 minutes 164 minutes 138 minutes

After the relevant files are downloaded and the installation process is completed, the

configuration of the various SolarWinds products varies slightly, due to their individual purposes.

One issue that was quickly apparent in this study is that SolarWinds requires four separate

products to meet the functionality found in the single instance of CA Nimsoft Monitor. Rather

than having to load, navigate, and analyze data from four separate products – none of which are

integrated together for dashboard analysis or reporting purposes in the trial versions – CA

Nimsoft Monitor offers a single, unified architecture and dashboard. It is this type of easy-to-use

navigation that gives CA Nimsoft Monitor a marked advantage over SolarWinds.

CA Nimsoft Monitor vs. Nagios XI

Nagios XI is an open-source IT monitoring product that runs on Linux servers. In a similar

approach to SolarWinds, Nagios offers a free 60-day trial period with a fully functional license for

Nagios XI. After 60 days, the user must decide whether to pay a licensing fee to keep Nagios XI

functional. Nagios XI is licensed on a per-monitoring-server basis. License pricing is determined

http://www.ca.com/snap

8 | P a g e

by the number of hosts (nodes) that the user intends to monitor. There are no restrictions on the

number of services that can be monitored with XI.

While Nagios XI is a relatively inexpensive solution, an experienced IT administrator may be

needed for its successful deployment and management. Downloading Nagios XI is a multi-step

process that lacks the guided process offered by CA Nimsoft Monitor. After installation, the

configuration process is highly technical, requiring the manual development of discovery queries

and manual alert generation without on-screen guidance. It took almost three and half hours

(197 minutes) to download, install, configure, and discover 100 devices with Nagios XI, while the

same routine took 61 minutes in total for CA Nimsoft Monitor 7.5.

Table 2: CA Nimsoft Monitor 7.5 vs. Nagios XI

CA Nimsoft Monitor Version 7.5

Nagios XI

Download 15 min 16 min

Installation 15 min 22 min

Configuration 16 min 118 min

Discovery 15 min 41 min


61 minutes 197 minutes

CA Nimsoft Monitor vs. IBM SmartCloud Monitoring

IBM SmartCloud Monitoring is available in a 30-day trial version that users can review and test.

IBM SmartCloud includes IBM Tivoli Monitoring and the following IBM Tivoli Monitoring for

Virtual Environments components:

Linux Kernel-based Virtual Machines agent

NetApp Storage agent, Network Devices agent, and VMware VI agent

Virtual Environments Dashboard

Virtual Environments Performance and Capacity Management Reports

Capacity Planner for Virtual Environments.

The download of SmartCloud Monitoring is completed by downloading two separate files, both

of which first require the download and use of IBM’s Java-based IBM Download Director. The

first of these two files is the SmartCloud platform, and the second is the infrastructure node

application. If users do not already have IBM Download Director installed, this requires a separate

download of the file management platform.

Like the download processes for SolarWinds and Nagios, the download of IBM SmartCloud

Monitoring is a highly manual, unguided, and unsupported process. The download and

installation of IBM SmartCloud was completed in 35 minutes in this study, while the entire

9 | P a g e

configuration process took two hours to complete. The total time from download to discovery

and monitoring of 100 devices was three hours (180 minutes).

Table 3: CA Nimsoft Monitor 7.5 vs. IBM SmartCloud Monitoring


IBM SmartCloud Monitoring

Download 15 min 15 min

Installation 15 min 20 min

Configuration 16 min 120 min

Discovery 15 min 25 min

Total Time to Monitor 61 minutes 180 minutes

Remote Monitoring Considerations for Distributed Environments

The ability to monitor remote networks is a growing need among MSPs and enterprises that need

to observe remote networks over the public Internet. MSPs typically offer “remote monitoring”

services for their clients, which require the ability to monitor the availability and performance of

the infrastructures within their customers’ internal firewalls. Many corporations also need to

monitor remote branch offices that are not connected to their corporate backbones.

While all of the solutions evaluated in this report are able to monitor IT elements within a local

environment, the ability to monitor remote customer networks under a single monitoring

domain varies among vendors. In order to achieve remote monitoring visibility, any solution will

require network connectivity to communicate with client devices and systems. Additionally,

monitoring tools are needed to understand Network Address Translator (NAT) and duplicate

private IP addresses from customer to customer. Finally, customer firewalls need to be

considered, as many companies want their data to be transmitted securely and with a small

footprint (i.e., minimal to zero firewall ports opened). As part of this report, the vendor products

included in this analysis were assessed in order to understand their remote monitoring

capabilities and ability to best address the needs of MSPs and large enterprises.

CA Nimsoft Monitor

CA Nimsoft Monitor includes deep

remote monitoring capabilities, which

were developed in the earliest

versions of the product. Customers

that need to monitor remote

locations, offices, and networks can

do so by installing a software-based

“hub” inside the client network.

CA Nimsoft Monitor Multi-Site Deployment

10 | P a g e

The CA Nimsoft Monitor hub then securely communicates monitoring information from the

remote site to the primary Nimsoft server, over a single port using an SSL tunnel. No third-

party VPN technology is required, and CA Technologies does not impose any extra charge for

this connectivity capability.

Nimsoft portal access can also be granted to each client, so that customers only have access

to monitoring data from their local hubs, which enables MSPs to supply live content to their

clients under a single Nimsoft deployment. CA Nimsoft Monitor delivers multi-tenancy

support to MSPs that leverage these capabilities.

SolarWinds Network Performance Monitor

There are four options for enabling remote monitoring with SolarWinds NPM. These include:

Option 1: Centralized deployment – This involves the centralized deployment of NPM, and

then the utilization of VPN connections to each remote network. If no VPN connection exists,

NPM must be deployed at each site, with access made possible via a web console. This option

can be costly and can increase monitoring complexity for MSPs and large enterprises, which

have to use multiple VPNs for this arrangement.

Option 2: Distributed deployment – This option allows for the deployment of NPM at each

customer site. Data from each NPM deployment is then sent to and processed by a separate

SolarWinds module, called SolarWinds Enterprise Operations Console (EOC), at the MSP’s

NOC. This option is very complex for MSPs, and requires a standalone SolarWinds NPM

deployment for each customer. Additionally, the SolarWinds Enterprise Operations Console,

which is the software required to combine all of the remote installations into a single MSP

NOC view, starts at a price of US$4,995 per deployment.

Option 3: Hybrid approach – This option is used when the customer has implemented more

than one SolarWinds product with one product in a centralized model and the others in a

distributed model. Similar to option 2 above, SolarWinds EOC is used to connect all the

platforms together for data collection and analysis. MSPs considering this option may be

concerned over the lack of standardization across their customer base, and the need for an

individual strategy for each client.

Option 4: N-able – SolarWinds also recommends that customers with a highly distributed

architecture consider their new N-able product to avoid remote monitoring challenges. N-

able is a remote monitoring company that SolarWinds acquired.

CA Nimsoft Monitor has the advantage in this case, as a dedicated VPN connection is not required

for each case of communication to each remote site.

11 | P a g e

Nagios XI

Like SolarWinds, Nagios suggests a full-time VPN connection to remote locations as its preferred

option. The company also offers a “distributed monitoring” configuration option. However,

employing this option can become very complicated, as it requires an additional module called

the Multi-Nagios Tactical Overview System (MNTOS) monitoring aggregation tool. MNTOS allows

the Nagios XI user to setup an aggregated view of multiple Nagios monitoring servers, making it

the preferred tool for distributed monitoring environments (similar to the Enterprise Operations

Consoles’ role for SolarWinds).

It is suggested that users fully research their remote monitoring options with Nagios, in order to

understand the cost and effort required to monitor remote customer networks with the

platform. While Nagios’ open source offering is free of software licensing costs, those savings can

be quickly burned as a result of the human hours required to get the environment up and running

– and to keep it up.


The distributed monitoring capabilities within the IBM SmartCloud platform were the hardest to

determine during our testing. Because information on this metric was not readily available, a

detailed analysis could not be completed.

Summary of Testing Scenarios In addition to benchmark tests of the vendor platforms, Apprize360 created specific testing

scenarios that were applied to each vendor’s products. The full scenario testing results can be

found in Table 7, which begins on page 24 of this report. The following is a summary analysis of

the test results, which highlights the most impressive scenario responses and capabilities

assessed for each vendor’s product.

Configuration


One of the advantages of CA Nimsoft Monitor is the “agent-optional” approach. This allows the

user to access to both an agent-based or an agentless monitoring configuration, based upon their

specific needs and their environment. CA Nimsoft Monitor’s agent-less monitoring option

collects information from “black box” systems, such as hypervisors, SAN storage systems,

network devices, SaaS, and cloud environments where no agents are permitted. CA Nimsoft

Monitor’s agent-based monitoring provides spooling of performance metrics in disconnected

environments so no data is ever lost in outages.

12 | P a g e

CA Nimsoft Monitor uses what it calls a “robot” (analogous to an agent) to collect and

disseminate information about a managed device. Customers that want to leverage on-board

monitoring configurations install a lightweight robot on each managed device. These robots

provide the communication functionality required for a system to be monitored in a CA Nimsoft

Monitor domain. CA Nimsoft Monitor offers the “Automated Deployment Engine” (ADE) that

enables administrators to quickly and centrally distribute agents to remote target servers,

removing the need to manually install the robot binary onto each system directly.

CA Nimsoft Monitor then leverages one or more “probes” to monitor specific

components/technologies on a managed device. For example, one common probe, the CDM

probe, is responsible for monitoring CPU, disk and memory utilization on target hosts. Over140

probes are available, allowing users to manage the entire IT infrastructure, including servers,

storage, network devices, applications and databases as well as user response time monitoring

and data center power consumption. Probes can be easily deployed across an entire network via

a simple drag-and-drop interface—or programmatically in an automated fashion.

SolarWinds

SolarWinds NPM can monitor the performance of SNMPv1, SNMPv2, or SNMPv3-enabled devices

within a network. SolarWinds NPM uses either Network Sonar discovery and import or Node

Management in the NPM Web Console to discover objects for monitoring. To discover and add

a larger number of devices across a network, the Network Sonar and Network Sonar Results

Wizards are available. The web console provides Web Node Management to add individual

objects for monitoring.

After Network Sonar Discovery has populated the SolarWinds Database with the network objects

to be included in monitoring, node and volume information is passed to the Business Layer. The

Business Layer passes node and volume information to the Collector Polling Controller and

provides licensing information to the SolarWinds Information Service (SWIS). At this point, the

SolarWinds Collector Polling Controller creates the required polling jobs and then passes them

on to the Job Engine. The Job Engine performs the requested polling jobs and then passes the

results to the Collector Polling Controller. There, all polling results are placed into the Microsoft

Message Queue (MSMQ). Here, the Collector Data Prosssor pulls polling results from the MSMQ

and performs calculations and then inserts these results into the SolarWinds database.

Nagios XI

The Nagios XI Monitoring Wizard is used to configure and monitor a new device. After the device

is discovered, a separate wizard is run to configure monitored elements. Each device category

requires a separate wizard (see diagram). In addition, Nagios XI requires multiple tabs and open

windows to complete a task that other products can complete in a single screen. This makes

device configuration potentially time consuming and complicated to conduct.

13 | P a g e


IBM SmartCloud Monitoring is an agent-based platform that requires agents to be installed on

various devices and sub-components. IBM SmartCloud Monitoring includes agents for various

devices and systems except for storage systems. Storage agents must be acquired separately.

Overall, configuring and deploying IBM agents is a complex and multi-step process.

With IBM SmartCloud Monitoring, there are two types of agent deployments supported:

OS agent deployment from the installation image or using IBM command called “tacmd

createNode”. The tacmd createNode command also creates a node, the directory into

which not only the OS agent is installed, but also where any non-OS agents are deployed.

Non-OS agent (such as the DB2 for Linux, UNIX, and Windows agent) deployment using

the Tivoli Enterprise Portal GUI (for other non-OS agents) or the “tacmd addSystem”

command.

We discovered that remote deployment of agents is the easiest method for agent configuration

and installation. Remote deployment provides the ability to deploy agents across the

environment from a central location. At this central location, a repository is created to store the

agent images called the “agent depot”. The agent depot is then populated with the desired

installation images and then is verified. Once the depot has been verified to contain all the

desired images, they are distributed out in the network. All agents that will be deployed in the

network require their application support files that contain the agent-specific information, which

must be installed in the Enterprise host server.

Out-of-The-Box Dashboards


CA Nimsoft Monitor provides intuitive,

up-to-date portal views of monitoring

data.

The solution features a monitoring

portal that provides a complete view of

the systems and services that underpin

vital business services, whether those

are based on any combination of virtualized infrastructures, SaaS offerings, cloud-based services,

or outsourced environments. In addition, the solution offers multi-tenant capabilities that enable

services providers, or enterprises, to monitor and manage the infrastructures of multiple clients

or users.

Configurable dashboards offer end-users and service providers real-time, immediate access to

the monitoring data that matters to them. CA Nimsoft Monitor reports provide snapshots and

14 | P a g e

historical context for quality of service, performance and SLA compliance monitoring. In addition

dashboards can be tailored to specific users, groups and roles, enabling users to see any type of

data that matters to them in a single view. For service providers, dashboards can be branded and

customized for each end user. CA Nimsoft Monitor also supports out-of-the-box dashboards for

VMware health and performance, power consumption, Amazon Web Services, and data center

performance.

SolarWinds

SolarWinds NPM users can select out-of-the-box dashboards or can build their own custom

dashboards via drag-and-drop placement charts and reports. Dashboards are also dynamic,

allowing for the user to click and drill-down into additional data.

One area where we discovered some complexity is when integrating specific data into NPM

dashboards. For example, a NPM user is still required to log into the server and use the old

Windows application when configuring alerting.

Nagios XI

Reporting is area in which Nagios XI is inferior when compared to CA Nimsoft Monitor. Nagios

requires the use of a report writer to generate reports on specific devices, alerts, and

performance analysis. The Nagios XI report writer involves a multiple-screen, multi-step process

that takes time to set up and to use to generate reports. In addition, Nagios XI reports are heavily

focused on node availability. Users can run a “report summary” of specific host availability as well

as nine pre-canned available reports, including a summary, event log, notification log, and

bandwidth usage report.

Dashboards are another area in which CA Nimsoft Monitor has a comparative advantage over

Nagios XI. While the main Nagios XI dashboard provides information on overall performance

state, as well as alerts on performance issues, Nagios has no single, unified dashboard for

understanding IT health on one screen. Instead, Nagios utilizes multiple dashboards on multiple

separate webpages, which are categorized by the specific host or service group.


IBM SmartCloud Monitoring has a central dashboard called a “health dashboard”. The dashboard

is automatically sorted by severity of issue. In addition, users can click directly into specific tools

as well as drill down into the actual physical component. For example, an IT administrator may

be alerted to a storage problem on the dashboard and has the ability to drill down into the IBM

Tivoli monitoring tools for storage management and begin to address the issue from there. The

other positive aspect of this tool is that by placing your cursor over a highlighted topic or device

creates a pop-up in the window, giving the user immediate information about that device.

15 | P a g e

Service Level Agreement (SLA) Reporting


CA Nimsoft Monitor includes a robust SLA tracking and reporting module that monitors

operations and business service levels against SLA targets and forecasts violations with warning

alerts. This part of the Nimsoft solution provides a graphical interface for defining SLA

parameters, which can include the compliance period and operating periods, exclusion windows,

the SLA target compliance percentage and more. Web-based SLA compliance reports with drill-

down granularity are auto-generated and auto-distributed.

The CA Nimsoft Monitor SLA reporting and monitoring solution will also continuously perform

calculations to determine if the current period SLA is safely in compliance. It will determine if a

SLA breach is imminent should the problem or condition be allowed to persist. CA Nimsoft SLA

reports include color-coded SLA compliance/breach trend indicators with forecasted breach date

and time. Warning and critical level alerts can be generated when the percentage compliance

decreases to predefined thresholds.

It is important to note that any metric can be tracked for SLA reporting within CA Nimsoft

Monitor, including non-IT metrics that may be important to your organization. As an example, if

the number of users logged into an e-commerce site or average sales price per transaction

metrics were held in an external database, CA Nimsoft Monitor could collect such metrics and

analyze them for SLA purposes. This module can help eliminate time consuming manual

reporting.

SolarWinds

SolarWinds’ IP SLA Manager is a free add-on module to SolarWinds NPM that is designed to

identify site-specific or WAN-related network performance issues. SolarWinds IP SLA Manager

can help locate which devices on a network actually support IP SLA and automatically setup

operations for those devices. SolarWinds IP SLA also allows users to monitor WAN applications

by taking the performance-pulse of underlying network protocols, including DNS lookups, FTP,

HTTP, TCP connect, and UDP jitter, while continuing to monitor VoIP call paths.

Note that SolarWinds does not appear to offer a “global” SLA monitoring or reporting application

that allows customers to mix and match cross-domain key performance indicators.

Nagios XI

Nagios XI does not have SLA support out-of-the-box, making this a difficult solution to support

MSPs as well as internal IT organizations that have SLAs with internal business units or external

service providers.

16 | P a g e


IBM SmartCloud Monitoring lacks integrated SLA reporting. SLA reporting with IBM SmartCloud

Monitoring is accomplished through licensing add-on IBM products. Customers can add SLA

monitoring to their deployment through using the IBM Tivoli Service Level Advisor or by

deploying IBM Maximo. IBM Tivoli Service Level Advisor is a wizard-based platform that allows

users to set, monitor, and report on IT services that are not meeting defined service levels. IBM

Maximo SLA Manager helps to define SLA agreements for monitoring as well as create rules for

escalation procedures when SLA bounds are breeched. This add-on module supports a real-time

graphical UI to display key performance indicators and SLA performance.

Overall ease of use – Unified Views, Workflows, Alarms, Device Views, & Reporting


CA Nimsoft Monitor alarm management helps IT operations personnel manage the flood of

events that come in from the IT infrastructure by eliminating duplicate event signals and filtering

events according to operational or business priorities. The goals are to improve the mean time

to isolate and repair problems and to prioritize IT support efforts according to business process

value. Through the solution’s custom filters, customers can set alerts only when an exception

occurs, such as an outage, failure, or threshold breach. CA Nimsoft Monitor alarm management

helps IT organizations reduce the time it takes to troubleshoot problems by consolidating events

from various devices, servers and applications and providing the ability to assign alarms to the

right IT staff.

CA Nimsoft Monitor features a monitoring portal that offers a single role-based, intuitive view of

all monitoring data. This portal extends the multi-tenant capabilities of CA Nimsoft Monitor,

enabling service providers to effectively report service level agreement (SLA) status and

monitoring information to clients, while offering clear, executive-ready dashboards to display SLA

status for in-house stake holders. Out-of-the-box portlets and end user level customization allow

CA Nimsoft Monitor users to adjust and optimize their monitoring views.

SolarWinds

SolarWinds NPM reporting is generated through the “Report Writer” feature embedded within

the web console. Report Writer also allows customers to create custom network monitoring

reports using SQL. A variety of predefined reports are also included in the NPM deployment.

One area of deficiency is the area of ad hoc report generation. While there are a number of pre-

defined reports, generating an ad hoc report on specific devices or groups took time and several

different steps to generate. Reporting is also limited to non-graphical based tables.

17 | P a g e

Nagios XI

Reporting is area in which Nagios XI is inferior when compared to CA Nimsoft Monitor. Nagios

requires the use of a report writer to generate reports on specific devices, alerts, and

performance analysis. The Nagios XI report writer involves a multiple-screen, multi-step process

that takes time to set up and to use to generate reports. In addition, Nagios XI reports are heavily

focused on node availability. Users can run a “report summary” of specific host availability as well

as nine pre-canned available reports, including a summary, event log, notification log, and

bandwidth usage report.


Integrated analytics is area of strength for IBM SmartCloud Monitoring. IBM SmartCloud

Monitoring supports a dynamic, graphics-based UI, which includes integrated capacity planning

and optimization of cloud and virtual environments. Capacity planning is an area of strength for

this platform. IBM SmartCloud Monitoring includes “what-if” analysis that enables IT

administrators to model potential capacity scenarios to make informed decisions on future

expansion. For example, whether it is possible to consolidate from three virtual hosts to two, or

how many older hosts would be required to run desired workloads, versus upgrading to new

ones. Users can model these scenarios to determine the most efficient, cost-effective IT strategy.

18 | P a g e

Summary and Conclusions CA Nimsoft Monitor version 7.5 is an easy-to-use and quick-to-deploy IT monitoring solution. Of

the eight solutions assessed, CA Monitor version 7.5’s installation and configuration process

made it the easiest and quickest way to begin monitoring IT assets.

There are several factors that make CA Nimsoft Monitor version 7.5 unique:

1) Monitoring functionality is hosted in a single, unified platform. It takes four SolarWinds

products to provide the functionality offered by a single instance of CA Nimsoft Monitor.

2) CA Nimsoft Monitor supports a guided installation and configuration process. This offers

a deployment process that is quick and intuitive, allowing for the correct decisions to be

made during initial configuration.

3) CA Nimsoft Monitor had the fastest “time to monitor” (time from download, through

configuration, and to reporting on device monitoring), at 61 minutes. The closest

alternative was SolarWinds NPM at 113 minutes, which is almost twice as long as the time

for CA Nimsoft Monitor and is limited to only network monitoring (see Table 4 and Figure

1 below).

4) CA Nimsoft Monitor’s drag-and-drop reporting made deep device reporting simple and

fast.

5) Of the eight products analyzed, CA Nimsoft Monitor’s discovery process was the fastest

in identifying devices and beginning active device monitoring.

6) CA Nimsoft Monitor version 7.5 supports easier remote monitoring for MSPs and large

enterprise environments through a single, unified platform.

7) CA Nimsoft Monitor version 7.5 was the most responsive, and was capable of addressing

most of the testing scenarios without any additional add-on modules.

0

50

100

150

200

250

CA Nimsoft SW NPM SWStorage

SW VM SW SAM Nagios XI IBM SCM

Download Install Configure Discovery

Min

ute

s

Figure 1: Total Time to Monitor

19 | P a g e

Table 4: Comparative Summary - Total Time to Monitor

CA Nimsoft Monitor

v7.5

SolarWinds NPM

SolarWinds Storage Mgr

SolarWinds Virtualization

Mgr

SolarWinds SAM

Nagios XI IBM

SmartCloud Monitoring

Download 15 min 11 min 8 min 15 min 10 min 16 min 15 min

Installation 15 min 52 min 44 min 73 min 60 min 22 min 20 min

Configuration

16 min 25 min 38 min 44 min 33 min 118 min 120 min

Discovery 15 min 25 min 35 min 32 min 35 min 41 min 25 min


61 minutes 113 minutes 125 minutes 164 minutes 138 minutes 197 minutes 180 minutes

TABLE 5: CA Nimsoft Monitor (v 7.1 & 7.5) vs. Solarwinds

Topics Description CA NIMSOFT Monitor

Version 7.1 CA NIMSOFT Monitor

Version 7.5

Solarwinds NPM Solarwinds Virtualization

Manager Solarwinds SAM

Solarwinds Storage Manager

SCORE NOTES SCORE NOTES

SCORE NOTES SCORE NOTES SCORE NOTES SCORE NOTES

Installation

Total Download

5 25 min 5 15 min 4 11 min 4 15 min 4 10 min 4 8 min

Total Installation

5 19 min 5 15 min 4 52 min 3 73 min 4 60 min 4 44 min

Download Size

4 1.2 GB 4 1.1 GB 4 5.4 GB 4 1.42 GB VMware/

1.4 GB Hyper-V 5 640 MB 5

315 mb windows/408

mb Linux

Installation Difficulty

5 EASY; Wizard-

based 5

EASY; Wizard-based

3 Multiple Steps 3 Multiple Steps 3 Multiple Steps 4 Multiple Steps

but straight forward

Installation errors

5 Zero 5 Zero 5 Zero 5 Zero 5 Zero 5 Zero

Configuration

Total Time to Configure

5 18 min 5 16 min 4 25 Minutes 4 44 Min 4 33 Min 4 38 Min

Ease of configuration

5 EASY; Wizard-

based 5

EASY; Wizard-based

4 "Intelligent"

alert configuration

3 Highly Manual, time consuming

4 Manual, time

consuming 4

Manual, time consuming

Platform Ease of Use

Overall Ease of Use of Platform

5 EASY 5 EASY 4 MEDIUM; Prior experience with IT mgt needed

4 Multiple screens with dashboards

4 MEDIUM; Prior

IT mgt experience

4 MEDIUM; Prior experience with IT mgt needed

UI & Navigation

5 Dynamic drill-

down 5

Dynamic drill-down

4 Drop-down

menus 4

Drop-Down menus

4 Drop down

menus 4

Drop-down menus

Ease of Performance Reporting

5 EASY 5 EASY 4 MEDIUM 4

Data is there, overwhelming & difficult to scale

back

4 MEDIUM 4 MEDIUM

Visualize component performance

5 HIGH VISIBILITY 5 HIGH VISIBILITY 5 HIGH VISIBILITY 4 Can force view

into graphic format

5 HIGH VISIBILITY 5 HIGH VISBILITY

Performance

Overall app performance

5 HIGH

PERFORMANCE 5

HIGH PERFORMANCE

5 HIGH

PERFORMANCE 5

HIGH PERFORMANCE

5 HIGH

PERFORMANCE 5

HIGH PERFORMANCE

Discovery of 100 network components

5 19 MIN 5 15 MIN 4 25 min 4 32 min 3

35 min; Requires separate module

4 35 min

Overall dashboard performance

5 HIGHEST

PERFORMANCE 5

HIGHEST PERFORMANCE

4 HIGH

PERFORMANCE 4

HIGH PERFORMANCE

4 HIGH

PERFORMANCE 4

HIGH PERFORMANCE

Diagnosis and Resolution

Performance issue identification

5 IMMEDIATE 5 IMMEDIATE 4 Slight Delay 4

Took 5 min before data storage capacity was

alerted

4 IMMEDIATE 4 IMMEDIATE

21 | P a g e

Problem Discovery Workflow

5 STRONG 5 STRONG

2 PARTIAL in

COMPARISON 5

Provides recommendations to remediate alert

2 PARTIAL in

COMPARISON 2

PARTIAL in COMPARISON

Level of Unification

Level of Integration

5 Unified platform 5 Unified platform 1 4 separate platforms

1 4 separate platforms 1

4 separate platforms

1 4 separate platforms

Reporting

OOTB vs. Customization

3 Capability of

Configuration 4

Deep configuration

capability 4

Requires custom report

writer 4

Requires custom report writer 4

Requires custom report

writer 4

Customer report writer

Quantity of Reports

5 HIGH NUMBER

OF REPORTS 5

HIGH NUMBER OF REPORTS

4 Several pre-

canned reports 4

Canned reports available; lots of

data 4

Several pre-canned reports

3 Tabular reports

Dashboards 5 SINGLE

DASHBOARD 5

SINGLE DASHBOARD

5 OTTB

Dashboards 4

Includes Benchmarks; Too much data on one

screen

5 OTTB

Dashboards 3

Static dashboards

Custom Report Capability

1 Limited

Customization 4

Deep Customization

5 Custom report

writer 5

Custom report writer

5 Custom report

writer 5

Custom report writer

Mobile Optimized Reporting

1 Limited

Optimization 5 Highly Optimized

2 Limited

Optimization 2

Limited Optimization

2 Limited

Optimization 2

Limited Optimization

Overall Self-Guided Support

Guided Supported during install & configure

5 STRONG Wizards

5 STRONG Wizards 3 Limited

guidance 3 Limited guidance 3

Limited guidance

3 Limited

guidance

Support Community

Support Community

3 Growing

community 4

Growing community

5

Large & extremely

active 'Thwack' support

community

5

Large & extremely active 'Thwack'

support community

5

Large & extremely


community

5

Large & extremely


community

Overall Score Out of 120 pts 107 116

93 92 93 92

Percentage Grade

89% 97% 78% 77% 78% 77%

22 | P a g e

TABLE 6: CA Nimsoft Monitor 7.5 vs. Nagios XI and IBM SmartCloud Monitoring

Topics Description CA NIMSOFT Monitor Version 7.5 Nagios XI


SCORE NOTES

SCORE NOTES

SCORE NOTES

Installation

Total Download time 5 15 min 5 16 min 5 15 Min

Total Installation Time

5 15 min 5 22 Min 5 20 min

Download Size 4 1.1 GB 5 255 MB MSFT VM 4 776 mb (2 files)

Level of Installation Difficulty

5 EASY; Wizard-based 3 Multiple Steps 3 Multiple steps; requires IBM

Download Director

Total number of installation errors

5 Zero 5 Zero 5 Zero

Configuration

Total Time to Configure

5 16 min 2 118 Min 2 120 min

Ease of configuration 5 EASY; Wizard-based 3 Requires manual query

development; manual alert generation without guidance

3 Requires manual alert generation

without guidance

Platform Ease of Use

Overall Ease of Use of Platform

5 EASY 4 Fairly easy to use 4 MEDIUM; Prior experience with IT

mgt needed

UI & Navigation 5 Dynamic drill-down 3 Drop-down menus with limited

dynamic drill-down 3 Drop-down menus

Ease of reporting performance of components

5 EASY 3 Requires a report writer, multiple

steps, slow report generation 3

Requires a separate report generation writer

Ability to visualize performance of components

5 HIGH VISABILITY 3 LIMITED VISIBILITY of all

Components 3

Limited visibility; requires multiple screens

Performance

Overall app performance

5 HIGH PERFORMANCE 4 RESPONSIVE 4 RESPONSIVE

Discovery of 100 network & system components

5 15 MIN 4 41 min 4 25 min

Overall dashboard performance

5 HIGH PERFORMANCE

4 Limited dashboard but

responsive 4 Multiple dashboards

Diagnosis and resolution

Performance issue identification

5 IMMEDIATE 2 Slow to identify; Example took 10

min for alerting 2

Slow to identify; Example took 15 min before alert generation

Guided workflow for problem discovery

5 STRONG 2 LIMITED 2 Limited guided workflow

23 | P a g e

Level of Unification

Level of Integration 5 Unified platform

2 Requires additional modules

(Fusion, XI, & Core)

2 Separate modules

Reporting

OOTB vs. Customization

4 Deep Configuration 2

Uses report writing engine for all reports; requires multiple steps

to generate a report; slow report gen

3 Limited customization; Option for

IBM BI integration

Quantity of Reports 5 HIGH NUMBER OF REPORTS 2 Uses report writing engine for all

reports 4

Uses separate report writing module; Several pre-canned reports; Supports capacity

planning

Dashboards 5 SINGLE DASHBOARD 3 Limited DB to include System

beck & behavior 3

Stat dashboards; No dynamic drilldown

Custom Report Capability

4 Deep Customization 3 Limited custom reporting 3 Limited customization; Option for

IBM BI integration

Mobile Optimized Reporting

5 Highly Optimized 1 Extremely limited optimization 1 Extremely limited optimization

Overall Self-Guided Support

Guided Supported during install & configure

5 STRONG Wizards 3 Limited guidance 3 Limited guidance

Support Community

Support Community 4 Growing community 5 Strong support community 0 No specific community for

SmartCloud Monitoring

Overall Score Out of 120 points 116 78 75

Percentage Score 97% 65% 63%

24 | P a g e

TABLE 7: Comparative Assessment of Response to Specific Scenarios

Specific Scenario CA Nimsoft Monitor Solarwinds NPM Nagios XI IBM SmartCloud Monitoring

Agent-installation Process Installed via a “robot”; Bulk robot installation via Automated Deployment Engine (ADE)

N/A - NPM is an agentless-only platform

NCPA agent (Nagios Cross Platform Agent) via Nagios Monitoring Wizard; requires manual input of IP addresses

Requires deployment of specific device/environment agents; Requires “Agent Builder” module to deploy & manage agents

Configure basic system monitoring of CPU, disk and memory via agent approach.

Uses robot for CDM probe; Can set polling frequency & number of samples for each component

N/A - NPM is an agentless platform Part of auto-discovery process; Displays info within “Service Status” dashboard

Requires separate module called “Agent Builder” to deploy & configure OS & CPU-specific agents

Configure basic system monitoring of CPU, disk and memory via an agentless monitoring approach

Uses RSP probe via SSH, WMI, or telnet protocol; Can set alerts on various thresholds

Graphic-based “Gauge” displaying memory load & utilization with ability to set alert thresholds

N/A – is an Agent-Only platform N/A – is an Agent-Only platform

Configure basic "ping" monitoring Use Net Connect probe; Integrated Ping sweep via IP addresses

Must configure through the NPM polling engine & set default polling settings; Requires separate PingSweep add-on (part of Solarwinds “Engineer Toolset”); uses SNMP, ICMP, & WMI polling

Part of auto-discovery wizard or can monitoring individual device if IP address is known

Requires “Agent Builder” module (sits outside platform) to configure basic monitoring; Not integrated into platform

E-mail notification, escalation and custom alarm management

Can generate a "business" alarm upon receipt of other alarms. Completed via the Nimsoft “NAS” and the Auto-operator

Free “Alert Central” module for central alert management; Includes desktop alert management

Basic email notification with limited alarm customization

Wizard-based; easy to setup with escalation management based on rules engine

Test a URL (via an HTTP test) and a ping test from a managed server

Requires URL Response Probe; Can test within Probe profile tab

Requires separate HTTP Monitor module for URL testing

Within dashboard; Has website URL monitoring wizard

Not able to test; Did not appear to support URL and ping test

Review the entire library of out of the box Unified Views/Dashboards that ship with the solution

24 dashboard views; all highly graphical dashboards that are customizable – can use “portlets” (widgets) to create custom dashboards for specific users or teams.

Can use “widgets” to create custom dashboards; Comprehensive dashboard capabilities within NPM with 30 pre-configured dashboards

30 dashboards by specific category; Custom “dashlets” priced separately; Has central “service status” dashboard

Comprehensive set of dashboards & reports with integrated capacity planning

Create a simple SLA report to track several metrics. Generate an SLA alert

Custom or pre-configured SLA reports; Performance on availability, frequency, response time, and quality; Provides a graphical interface for defining SLA parameters

Requires additional module to be licensed; Solarwinds IP SLA Monitor or Solarwinds Network Quality Manager required for SLA reporting

Limited SLA reporting support; Basic up-time reporting only

Can set and track limited SLAs from Health dashboard; Requires add-on modules from IBM Service Level Manager for more in-depth SLA monitoring and reporting

Overall ease of use, unified views, workflows, built in alarms, device views in a single interface

Single unified interface with one view of device status, alarms, and workflow execution

Multiple tabs & apps required; Customers may have NPM and SAM deployed along with other modules.

Multiple tabs and webpages required for different views of devices & device categories

Integrated unified interface; Contact-sensitive interface with dashboards

A Special Report by Apprize360 Intelligence for CA Technologies, Inc. All Rights Reserved. Copyright 2014.

Apprize360 Intelligence, INC. www.apprize360.com San Francisco, CA [email protected]

http://www.apprize360.com/

Comparative Analysis of IT Monitoring Tools

Technology

Transcript of Comparative Analysis of IT Monitoring Tools