A Practical Vulnerability Assessment Program

1520-9202/07/$25.00 © 2007 IEEEP u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y36 IT Pro November ❘ December 2007

V U L N E R A B I L I T Y A S S E S S M E N T

A PracticalVulnerabilityAssessment Program

B efore the Internet’s widespread use,which began in the late 1980s, comput-ing was centered almost completelyaround mainframe computers.Security

measures were typically limited to access controlsbetween local users and the data stored on localcomputers. The term “vulnerability” was uncom-mon, and few if any automated applicationsexisted for vulnerability detection.This was largelybecause system and software manufacturers rarelyreported such flaws to system owners, and evenmore rarely to the general public, simply becausethese vulnerabilities weren’t exposed to the out-side. Today, computer networks are so intercon-nected that a stranger can contact your PC fromthe other side of the world in milliseconds. Insteadof annual vulnerabilities reported in single digits,they’re now reported in the thousands and aredoubling every year, as Figure 1 illustrates. Themost sobering fact is that the average time it takesto exploit a newly reported vulnerability is just a

few hours from the vulnerabil-ity’s release.

To protect their critical ITassets, most enterprises usecombinations of technical pro-tective and detective solutionssuch as firewalls, malicious code

screening servers,and intrusion detection systems.Properly placing infrastructure security solutionscan increase the effectiveness of an overall enter-prise security profile,but technical point solutionsalone won’t provide a comprehensive securitystrategy. For an enterprise to identify susceptibil-ity to attacks before their IT systems areexploited, it must also perform regularly sched-uled vulnerability assessment and remediation.

Many enterprises perform IT vulnerabilityassessments as ad hoc tasks rather than ongoingbusiness processes. This type of piecemealapproach typically results in inadequate evalua-tion of the overall enterprise security profile andoften creates a false sense of security.We describea practical and repeatable methodology to man-age the vulnerability assessment program (VAP)and a set of vulnerability detection and remedia-tion practices to effectively implement and main-tain a VAP life cycle.

KEY PLAYERS IN A VAPA practical VAP is more than security hard-

ware, software, and staff support. It requires play-ers at different levels and from different areas ofthe enterprise.

Chapter 1, page 1, of every book on enterprisesecurity states, “A security program must have

Firewalls, malicious-code-screeningservers, and intrusion-detection systemsaren’t sufficiently comprehensive to ensureappropriate and ongoing information con-fidentiality, integrity, and availability. Apractical vulnerability assessment programlets organizations identify potential securi-ty exposures and correct any deficiencies.

Internet SecurityResources

Inside

Simon Liu, Larry Holt, and Bruce Cheng

November ❘ December 2007 IT Pro 37

senior management support to be successful.”A top-downapproach to a successful VAP starts with the chief execu-tive officer (CEO).

For any individual IT program element to be adoptedeffectively, it must be propagated by an individual who hasthe authority to speak for the CEO on all IT-related mat-ters.This is the task of the senior executive responsible forIT, or the chief information officer (CIO).

One individual must have the authority and vision tomake information security decisions, and be responsiblefor managing security-related program elements. This isthe information system security officer’s task.

Security administrators should have a wide range of spe-cialized skills so they can effectively perform the multifac-eted elements of enterprise-wide vulnerability assessment,vulnerability data analysis, remediation assistance,and fol-low-up baseline audits.

Chances are that the security administration staff won’tbe large enough, or have the permission, to physicallyremediate known vulnerabilities in every system through-out the enterprise. Instead, the existing systems adminis-tration staff can typically implement the necessary systemchanges more rapidly and on a much wider scale.Additionally, systems administrators are more likely toknow the intimate details of how a given system’s hard-ware, operating system, and applications interact. Thislevel of knowledge provides an additional fail safe to pre-vent any inadvertent interruption of normal businessprocesses.

MULTIFACETED VULNERABILITY ASSESSMENT How can systems and network administrators evaluate

their own security’s state? How can management be surethat their systems and networks can withstand attacks?

One suggestion is to know what your enemy knows aboutyou. Simply put, your vulnerability assessment should fol-low the same basic methodologies used by would-beintruders, as Figure 2 on the next page illustrates. Youshould assess vulnerability from remote locations toreveal vulnerabilities that are visible from outside yourenterprise (and within), and to review vulnerabilities thatauthorized users could potentially�either purposefullyor accidentally�exploit. Keep in mind that although thetwo assessment vectors differ, the attack methodologyaxioms remain the same.

Discovery scanning Information leakage from your enterprise is the first

information source an attacker attempts to exploit. Inmost cases, this is a completely legal endeavor on theintruder’s part. Improperly configured systems can offerseemingly trivial information, such as user names thatcould give a would-be attacker information on internalnaming conventions, or personally identifiable informa-tion about key IT personnel that an attacker could use tofalsify identities.

When performing a discovery scan, you shouldn’t useany known information other than the target organiza-tion’s name. You can find Web sites via public searchengines and identify IP address ranges from public WhoIsdatabases.You should attempt DNS zone transfers to seeif systems will give up internal DNS naming conventions.And finally, you should crawl public Web sites to find allunintentionally advertised private information aboutusers, systems, networks, and applications. Several Swissarmy knife (all-in-one) scanning tools are effective for dis-covery scanning. Examples include WSPing Pro andSamSpade, a publicly available freeware application.

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Q1–Q22007

9,000

8,000

7,000

6,000

5,000

4,000

3,000

2,000

1,000

0

Figure 1. Vulnerability trends, from 1995 to early 2007.

38 IT Pro November ❘ December 2007


Network scanning After discovering what publicly accessible networks exist

in your enterprise, an attacker’s next step is to map outremotely accessible hosts in your enterprise network.Youshould perform ping sweep scans to create a list of systemsthat acknowledge presence, whether they’re internal orexternal to the organization, as long as they’re reachablefrom the Internet. Additionally, network scanning cansometimes reveal the vendor brands of systems being used,as well as operating system types and versions. Again,although scanning other people’s systems isn’t proper“netiquette,”discovering publicly accessible hosts typicallydoesn’t break any federal or state laws. So, the intruderrisks nothing from this type of effort.

You should approach a network scan without any pre-conceived ideas of what systems exist within your enter-prise. For most sites, it’s not uncommon to find roguesystems and services that someone has activated withoutthe security or systems administrator’s knowledge.Established network scans should also occur pseudoran-domly, instead of sequentially across a range of networkIP addresses. Pseudorandom scans let you scan a networkundetected. Furthermore, only the security administratorshould know when scans are scheduled and what types ofscanning tools will be used, so system administrators don’tintentionally prepare a host to be scanned, and thus com-promise the scan’s results. Network Mapper (Nmap), apublicly available freeware application, is the most widelyused network-scanning tool.

Host scanning After identifying network-

accessible hosts, an attacker willwant to determine what servicesare available for potential ex-ploitation through known vul-nerabilities. An attacker couldfind most vulnerabilities in anenvironment by running the net-work scan results through thou-sands of exploit tests on eachnetwork-accessible host usingautomated tools or scripts.

Many enterprises stop at net-work scanning because host-levelscanning is a labor-intensive andtime-consuming process. Hostscanning typically runs a special-ized application or script thatloads the most current vulnerabil-ity signature (based on known vul-nerability databases) to rapidlyand automatically test each net-worked host against its listeningport (that is, the port inside a sys-tem that responds to inquiries

from other systems). Additionally, host-level scans usingknown vulnerability exploit methods can slow down, or insome cases shut down, a system’s ability to deliver busi-ness-related services. For some enterprises, performingrandom audits is more practical than running host-levelscans on every system.So, they run host-level scans againstselected representative systems to locate potential con-figuration errors that might span more systems, therebyintroducing vulnerabilities that put all systems of the sametype at risk.

Assuming that attackers won’t compromise an internalsystem left in a “soft” state is a dangerous gamble thatcould result in a single system compromise becoming alarge-scale multisystem exploitation.For this reason,enter-prises should perform host-level scanning�using, forexample, the publicly available Nessus or Tara for Unixsystems�to monitor and improve system configurations.

Application scanning Most attackers who’ve managed to compromise one or

more of your systems from a remote location won’t takethe time to run an application-level scan. If they exploiteda host vulnerability giving them administrative or rootaccess to the system, they wouldn’t need to. But for lower-privilege account compromises, or attempts made by alegitimate internal user wanting greater access rights,host-level scans can point to configuration flaws that canpotentially bypass host-level protection and allow privi-leged access.

Discovery scanning

Network scanning

Host andapplication scanning

Compromise

Wirelessscanning

Analogscanning

Figure 2. Multifaceted system vulnerabilities.


tially sensitive transmissions that are unencrypted.Again,your assessment team can easily do the same types of scansproactively to secure existing analog ingress points andidentify unauthorized modems. Telesweep is an excellenttool for these scans. However, if budget is a concern andyou can tolerate a little less functionality, the freewareToneLoc application, which has been around since the1980s, can still be effective.

VULNERABILITY-ASSESSMENT TOOLS From a technical standpoint, the first task in imple-

menting a VAP is selecting a set of tools for performingvulnerability scans. The basic cat-egories for such a standard tool kitshould cover the types of assess-ment that are planned.You shouldselect tools based on their price,performance, and reported qual-ity. To help secure your system,you should use the same tools anattacker uses to compromise yoursystem. The stereotypical hacker

mostly uses freeware, so you should consider using free-ware products in your tool kit.This isn’t to say that com-mercial off-the-shelf tools aren’t adequate. COTS toolsgenerally provide more comprehensive reporting capa-bilities than freeware tools. And they often providegreater capability, as well as the ability to interact withthe vendor to resolve any problems encountered with theapplication itself. But be prepared to use a combinationof freeware and COTS, because the final decision forwhat you use might rest on the enterprise security budget.Table 1 (next page) lists some popular scanning tools, bythe type of scanning they provide.

The amount of time it takes to assess your environmentdepends on the staff resources available and the enterprisenetwork’s size. This is particularly important when theenterprise imposes a specific scanning schedule, becausea complete scan can take well over 24 hours, depending onthe network size.Additionally, if the network or host capac-ities are limited, you should consider the impact on targethosts during the scan.An aggressive vulnerability scan canaffect production services.

Finally, you should consider the quality of reports thatcan be generated. Some tools generate concise results,while others generate telephone book-sized results thatbecome onerous to use. You should also be able to runexecutive summary reports without a lot of manual inter-vention. Although this type of report has little value toadministrators, you shouldn’t overlook it as an invaluabletool for demonstrating a return on investment.

VULNERABILITY ASSESSMENT POLICY The key to a successful VAP is, of course, having a pol-

icy. Before you run the first scan, you need a formal VAP

Individual misconfigured or poorly coded applicationsdon’t necessarily present themselves as known vulnera-bilities to network or host-level scanners. But an attackerwith a high degree of knowledge about the application canuse them just as effectively to compromise a system.

As with host-level scanning, many enterprises don’t useapplication scanning because it was once a labor-intensiveprocess that required manual testing or loading special-ized scanning applications directly on each system for theapplication being evaluated. Fortunately, specialized application-scanning-tool vendors are becoming moreaware of this threat as attackers are compromising moreWeb sites remotely through appli-cation-level exploitation.To answerthis need practically, the new breedof application scanners can simu-late attacks, such as SQL injectionfor applications that communicatewith back-end database servers.

A popular commercial applica-tion for this purpose is Watchfire’sAppScan tool. But even the free-ware scanner Nikto can sometimes detect Web applicationproblems based on known vulnerability signatures.

Alternate ingress scanning So you’ve implemented technical solutions such as fire-

walls and intrusion detection systems, and your vulnera-bility assessment includes regularly scheduled discovery,network, host, and application scanning and remediation.You’ve covered everything, right? Wrong. A determinedhacker can still pop right into the heart of your networkusing high- or low-tech means.

Many enterprises recognize wireless as the up-and-coming threat to their perimeter security profile. Using awireless-enabled laptop or PDA and a freely availablewireless scanner such as NetStumbler or Kismet, would-be intruders can pull up outside of your building with apowerful WiFi antenna and enter an improperly securedwireless access point. The good news is that your assess-ment team can easily do the same type of scan proactively,not only to secure existing wireless ingress points, but alsoto identify any unauthorized or rogue access points.

Conversely, many enterprises don’t think about thethreat of plain old telephone lines.They might not be fastor technologically advanced like modern routers,but givenenough time, attackers can use them as entry points todamage your enterprise just as they could through anyhigh-speed access point.Attackers can use a simple RadioShack device to convert even newer digital phone linesinto analog modem ingress points to your core networks.Additionally, even old or improperly configured faxmachines can make appealing targets for hackers.Unprotected fax lines can allow easy compromise of a faxmachine’s configuration, leading to interception of poten-

You should approach anetwork scan withoutany preconceived ideasof what systems existwithin your enterprise.


policy document that outlines the rules of engagement.Allsenior executives, business process owners, and adminis-trators must review and formally adopt this document toensure that everyone understands and agrees to what’sexpected of them. The policy document should clearlydefine where,when,and how the VAP will assess the enter-prise network, along with remediation expectations. Thedocument should

• define roles and responsibilities;• identify authorized scanning

systems;• define the tools to be used;• define the types of scans to be run;• define each scan type’s execution

frequency, time frame,and duration;• define the required remediation

time frames based on the vulnera-bility’s severity (for example, high,medium, and low);

• outline the assessment life cycle;and

• define the scan exclusion pro-cedure.

VULNERABILITY ASSESSMENT PROCESS

Vulnerability assessment is an itera-tive process,as Figure 3 illustrates.Thevulnerability assessment life cycleshould be a repetitive part of the over-all business process.

Definition phaseDuring the definition phase, you

ascertain the exact number of systemsin the enterprise, any network demar-cations that prevent complete accessfrom scanning systems, and any sys-tems that react adversely to normalassessment scanning tools. You con-duct definition-phase scanning usingdiscovery-scanning tools that map outthe enterprise network infrastructure’soverall landscape and perform safechecks against identified nodes. Thedefinition phase scanning’s adversereaction element not only ensures thatbusiness processes aren’t interrupted,but also instills user,administrator,andmanagement confidence in the vul-nerability assessment process.

Definition-phase scanning shouldtake place during typical workinghours, while load from normal opera-tions is at its peak.You should closely

coordinate this phase with systems administrators andprocess owners so they can monitor the system for any illeffects from scanning. You should remove any systemsfound to be adversely affected from scanning configura-tion profiles and include them on a formal exception list.

Normal scanning techniques shouldn’t affect healthy sys-tems and applications.Therefore, placing systems on a for-mal exception list doesn’t resolve the problem. Rather, it


Vulnerabilityalert

management

Vulnerabilityscanning and

analysis

Vulnerabilityremediation

Remediationaudits

Definitionphase

Baselinephase

Figure 3. Vulnerability assessment process (VAP).

Table 1. Network scanning tools.

Type of scan Tool COTS Freeware

Discovery WSPingPro X

Discovery SamSpade X

Network Internet security systems X

Network Network Mapper X

Network Nessus X

Host (Unix & Windows) ISS X

Host (Unix & Windows) Nessus X

Host (Unix & Windows) Tara X

Application AppScan X

Application Nikto X

Analog TeleSweep X

Analog TonLoc X

Wireless AirMagnet X

Wireless etStumbler X


temporarily excludes systems that are suscepti-ble to denial of service (DoS) caused by aggres-sive host scans that anyone, including un-authorized sources, could perform. You shouldreview susceptible systems and applicationsidentified during this phase for causes, such asmissing patches or inadequate hardwareresources. After identifying the problems, youshould assign each system an expected resolu-tion date, correct the problems, and rescan themduring the next scheduled assessment cycle.

Baseline phase If an enterprise doesn’t have a regularly sched-

uled assessment and remediation program, vul-nerability levels and numbers between organi-zational entities within an enterprise can differradically. This is especially true if different sys-tems administration teams maintain differentdepartmental systems within the enterprise.

To reach parity throughout an enterprise, eachsystem owner group should remediate systemsto a coarse minimum standard baseline withregard to legacy vulnerabilities. To derive thisbaseline, the enterprise should measure vulner-abilities by severity and number per system ineach organizational entity. Because scans mightfind many vulnerabilities in a system that’s neverbeen scanned before, you should relax remedia-tion timelines accordingly. At this stage, youshould encourage progress, but avoid over-whelming systems administrators and owners.

Security administrators should also use thebaseline phase to tune out as many false-positive results as possible from the scanningapplications being used.This will increase reportclarity and make it easier for systems adminis-trators to review future assessment reports. Insome cases, this might be as easy as changing thevulnerability signatures themselves. In extremecases, where signatures can’t be modified andvendor fixes can’t be requested, you’ll have toremove the signatures causing false positivesaltogether. However, you should do this only asa last resort.

After the definition and baseline phases, busi-ness process owners and administrators shouldbe reasonably comfortable with the vulnerabil-ity-assessment process. As we discussed earlier, at thisstage, the assessment time tables should begin to varypseudorandomly and remain confidential.During the auditphase,you should be able to detect some unauthorized sys-tems or services enabled by authorized users. But moreimportantly, randomized scans increase the likelihood thatan administrator will detect unauthorized system or serv-

ice accesses by intruders who target off-hour time framesto increase their chances of success and anonymity.

Vulnerability alert management Several commercial and federal sources, such as CERT

and US-CERT announce vulnerabilities daily (see the“Internet Security Resources” sidebar). Many of these

The CERT Coordination Center (http://www.cert.org) is acenter for Internet security located at the Software EngineeringInstitute at Carnegie Mellon University. The center studiesInternet security vulnerabilities, handles computer security inci-dents, publishes various security alerts, researches long-termchanges in networked systems, and develops security informa-tion and training.

The US-CERT (http://www.uc-cert.gov/federal) is a computeremergency readiness program at the Department of HomelandSecurity. The US-CERT established several collaborationgroups and programs to foster and facilitate information shar-ing on cybersecurity issues among government agencies.

The Computer Incident Advisory Center (http://www.ciac.org)is located at the Lawrence Livermore National Laboratory.CIAC provides solutions to US government agencies facingtoday’s IT security challenges. The center maintains core com-petencies for information protection through high-tech, inte-grated information security product development andconsulting services.

The Computer Security Institute (http://www.gocsi.com) is amembership organization that serves and trains the informa-tion, computer, and network security professional. Establishedin 1974, CSI provides educational opportunities and advocatesthe importance of protecting information assets.

The Information Systems Security Association (http://www.issa.org) is a not-for-profit international organization for infor-mation security professionals and practitioners. ISSA provideseducation forums, publications, and peer-interaction opportu-nities that enhance its members’ knowledge, skills, and profes-sional growth.

The Red Hat Security Resource Center (http://www.red-hat.com/solutions/security) provides open source solutions,including information, software tools, training, and consultingfor Linux system security.

The System Administration, Networking, and SecurityInstitute (http://www.sans.org) was founded in 1989. SANS is acooperative research and education organization through whichmore than 96,000 system administrators, security professionals,and network administrators share security information.

SecurityFocus Online (http://www.securityfocus.com) listsknown vendor vulnerabilities online to enable companies tomitigate risk, manage threats, and ensure business continuity.

Internet Security Resources


vulnerabilities won’t apply to systems or applications inyour enterprise. Paring these vulnerabilities down to a listof alerts that relate only to your operating system andapplication inventory is the vulnerability alert manage-ment element of your assessment program.Taking the timeto cross-reference vulnerabilities before promulgatingthem to systems administrative staff will make the reportsmore focused,easier to follow,and less likely to be ignored.

Vulnerability scanning and analysis The heart of the VAP is the running of regularly sched-

uled discovery, network, host, application, wireless, andanalog scans.You should tailor the frequency of each scantype to your security and systems administration staff’sability to not only run the assessments but also analyze thereports and remediate the discovered vulnerabilities in atimely manner.

Vulnerability reports for every tool using the default toolconfigurations will typically result in a higher than necessarynumber of false-positive results.Too many false positives canmake systems administration teams less willing to addressremediation in a timely manner.For that reason,you shouldanalyze assessment report results not only to identify validvulnerabilities, but also to eliminate false positives.

Vulnerability remediation Vulnerabilities should be corrected on a set time sched-

ule according to their severity and the technical staff’s abil-ity to perform the necessary corrections.For example,mostsites will require the staff to correct vulnerabilities with ahigh possibility of allowing remote administrative com-promise within a set number of hours,but correct low-levelvulnerabilities that allow only information leakage withina set number of days or weeks.

Remediation audits The most important part of any VAP is the remediation

of discovered vulnerabilities. You can measure the reme-diation effort by running follow-up scans and performingdifferential comparisons against previous assessmentreports. These remediation audits will provide solid met-rics to ensure that all areas of your organization are con-tributing equally to the VAP’s goal of improving theoverall security profile.

V ulnerability assessment’s future lies in an expansionof capabilities, an increase in the breadth and depthof coverage, and integration with other security and

operational technologies and processes. Major challengesinclude the volume and language of data output, contex-tual knowledge of the element being assessed, and multi-ple yet disparate assessment methodologies.

Vulnerability assessment tools can generate an over-whelming amount of data.To become operationally effec-tive in large enterprise environments, these tools mustprovide a correlated report of remediation steps per hostor groups of hosts in addition to an expanded list of vul-nerabilities. Current vulnerability-assessment tools lacknetwork-wide visibility. They don’t offer security mecha-nisms to protect assets from exploitation or provide visi-bility into the attack paths that can lead to an exploitation.This lack of visibility into the network-wide security pos-ture can limit the automation of prioritization activitiesand require extensive manual analysis to determine theremediation priority.

Current vulnerability-assessment technologies attemptto limit their impact on an environment by using nonin-trusive scanning, which can limit their accuracy in somecases. Future organizational security postures can benefitfrom structured and controlled penetration testing, whichcan provide a depth of coverage that many current vulnerability-assessment tools lack when run in an unin-trusive mode.

Organizations face an increasing array of assessmenttechnologies that specialize in specific elements of theirenvironment. We need tools to converge various assess-ment data sources, and to aggregate and correlate dis-parate assessment technologies to provide visibility intoan enterprise’s overall security state. Without this com-plete and correlated view of various assessment technolo-gies, it’s difficult for an organization to truly understandits security posture. Increasingly,vulnerability assessment’svalue will come from integrating with multiple securitytechnologies, improving report output quality, and moreeffectively protecting the IT environment. ■

Simon Liu is the director of information systems at the USNational Library of Medicine,National Institutes of Health.He is also an adjunct faculty member at Johns Hopkins Uni-versity. Contact him at [email protected].

Larry Holt is a security consultant at Computer SciencesCorporation. He was the leader of the IT security team atthe US National Library of Medicine, National Institutes ofHealth. Contact him at [email protected].

Bruce Cheng is a security consultant at Computer SciencesCorporation. He is currently the leader of the IT securityteam at the US National Library of Medicine,National Insti-tutes of Health. Contact him at [email protected].


A Practical Vulnerability Assessment Program

Documents

Transcript of A Practical Vulnerability Assessment Program