Post on 26-Mar-2015
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Brian MartinJake Kouns
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
•Overview•Inherent Problems•Important Issues •Major Players•Research and Rankings•Future
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Overview
© Open Security Foundation 2005
Vulnerability Database Vulnerability Database OverviewOverview
• What is a Vulnerability Database (VDB)?
• Database of information on security vulnerabilities. Simple!
• What about “dictionaries” (CVE) or “searchable indexes” VDB!
• Key is realizing VDBs will have their focus– Comprehensive Vulnerability Database – Focused Vulnerability Database – Vulnerability Notification Services– Value Added Services
© Open Security Foundation 2005
Brief HistoryBrief History• First VDBs were private, mostly maintained
by hackers or budding security geeks (before security professionals were common)
• First public database?– Unix Known Problem List– Internal Sun Microsystems Bug List– Early CERT database
• VDBs abandoned (Fyodor), sold to corporations (BID), or home grown (X-Force)
• Additional VDBs continued to be launched to meet different demands (Secunia, OSVDB)
© Open Security Foundation 2005
Basics of a VDBBasics of a VDB
• Vulnerability information gathered• Identification number/name assigned• Adherence to standard format• Ability to search and display dataOptional:• Mail lists (private or public)• Exports for integration• Other services
© Open Security Foundation 2005
Purposes of a VDBPurposes of a VDB
• Provide accurate information on security vulnerabilities
• Provide historic reference on software bugs
• Provide information on solutions • Provide innovations to help
organizations deal with vulnerabilities
But are they?
© Open Security Foundation 2005
WIIFM – What’s in it for WIIFM – What’s in it for me?me?
• Alerting/Notification– Information provided in timely fashion
• Detailed Content– Concise description, additional
analysis, references
• Organized Information– Vulnerability statistics– Trending – Historical context
© Open Security Foundation 2005
Vulnerabilities TrendsVulnerabilities Trends
CERT CERT VulnerabilitVulnerability Countsy Counts((1995-2004)1995-2004)
© Open Security Foundation 2005
Who uses a VDB?Who uses a VDB?
• Administrators• Auditors• Security Testers
– Penetration Testing– Vulnerability Assessments– Risk Management
• Criminals– Hackers, Crackers, Blackhats,
Greyhats, OH MY!
© Open Security Foundation 2005
Legalities and LiabilityLegalities and Liability
• Issues with disclosure– Bug finder and irresponsible disclosure– Do VDBs have a responsibility to be ethical for bug
finders?
• Liability for providing information– Liability for including exploit code?
• Copyrights on information– Including unedited original source?– Re-branding or re-writing?
• Confusing lawsuits– Tegam vs. Guillaume Tena (France)– Sybase vs. NGSS? (US)– HP vs. NGSS? (US)
© Open Security Foundation 2005
VDB SociologyVDB Sociology
• VDBs are taken for granted by users• Users need them but do not appreciate• Users rely on a VDB for 'thoroughness',
when they usually are not• Users quote VDB information as gospel,
as if VDBs confirm and validate every entry
• Users typically have favorite VDB, and only use that one
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Inherent Problems
© Open Security Foundation 2005
Inherent Problems with Inherent Problems with VDBsVDBs• Dependency
– If no entry for Product X, assumption it is secure
– Assume information is accurate, becomes gospel– Rely on VDB to alert you?
• Lack of Updates– Hard to update old entries (why don’t new
players care about old entries?)– Solutions not there or not fully updated– Workarounds not accurate or helpful
• Thoroughness– “multiple” entries– No digging for details– Ignoring obscure products
© Open Security Foundation 2005
• Lack of standard– Naming conventions– “multiples” vs. breaking out entries– What deserves an entry at all
• Accuracy and Integrity– Who updates? What motivation to be accurate?– Myth/Fake– Why is the information inaccurate?
• Poorly written advisory, Lousy research• Poor vendor communication/verification
– Why do VDBs trust anything and everything they read?
• Number of database entries matter
Inherent Problems with Inherent Problems with VDBsVDBs
© Open Security Foundation 2005
Inherent Problems with Inherent Problems with VDBsVDBs• Pros & Cons of adding entries
– Fast• No external references• Incomplete or inaccurate information
– Slow• Not timely like many people want
• Statistics & Metrics– Lack of classification (leads to problems)– Lack of severity (debate unto itself)
• Not only based on remote vs. local …• Availability of exploit• Impact of exploit• Installation base of software
© Open Security Foundation 2005
Inherent Problems with Inherent Problems with VDBsVDBs• Relying on Bug finders
– Double edge sword, good bug hunters provide great information, many do not
– Vulns being reported although previously disclosed
– Not including versions or vendor site, and not easily Google'd
– Vague information, untested – Advisories without dates (big vendors
especially guilty.. MS, IBM, Novell, Sun, HP)
– People try to use bug finding as a way to advertise their security services
© Open Security Foundation 2005
Inherent Problems with Inherent Problems with VDBsVDBs• What else?
– Many don’t make database easily available in full or not portable
– Don’t support third party utilities and use
– VDB snobs, refuse to reference certain other databases or sources
– Narrow focus on where to find vulnerability information (life outside Bugtraq)
– Often don’t give credit where due– […]
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Important Issues
© Open Security Foundation 2005
Important Issues for Important Issues for VDBsVDBs
• Most issues are easily overlooked• 7 key issues for a VDB to address
•User Dependency•Content Updates•Content Depth•Standards•Accuracy and Integrity•Statistics and Metrics• Integration Ability
© Open Security Foundation 2005
User DependencyUser Dependency
•Can you rely on a VDB?•Do you verify the VDBs
statements?•Do you read into the
information and make assumptions?
•Rely on VDB to alert you?
© Open Security Foundation 2005
Content UpdatesContent Updates
• Turnaround on new entries• Older entries need attention
– Updated external references– Updated solutions– Updated information on risk ratings
• Do all VDBs care about older entries?
• Corrections to entries
© Open Security Foundation 2005
Content DepthContent Depth
• Number of entries– Catalogue all vulnerabilities or just major
issues• Vague information on vulnerabilities
– Often due to poor research or vendor not providing details (thus, external references are important)
• Effort to correlate or research– Weeding out duplicate entries
• Types of products cataloged– Not just about Windows and Unix anymore
© Open Security Foundation 2005
StandardsStandards
• Definition of a Vulnerability• Naming Conventions• Dates• Write-ups• Risk Ratings• Solutions
© Open Security Foundation 2005
Accuracy and IntegrityAccuracy and Integrity
• Who maintains the data • How are updates justified• Motivation for entries• Motivation for accuracy
© Open Security Foundation 2005
Statistics and MetricsStatistics and Metrics
• How many entries exist?• How many entries are missing?• How do we know?• How many entries have solutions?• How many are critical?• How many vulns per month/year?• How many vulns per
vendor/product?
© Open Security Foundation 2005
Integration AbilityIntegration Ability
• Can users change or ask for updates
• Is the data easy to obtain• Does the VDB support 3rd parties• Does the VDB reference all
information• Can users dynamically pull
information
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Major Players
© Open Security Foundation 2005
Major PlayersMajor Players
• Comprehensive VDBs– BID - http://www.securityfocus.com/bid– CVE - http://www.cve.mitre.org/– ISS X-Force - http://xforce.iss.net/– OSVDB – http://www.osvdb.org/– Secunia - http://www.secunia.com/– Security Tracker -
http://www.securitytracker.com/• Vulnerability Notification Services
– CERT - http://www.cert.org/– CIAC Advisory -
http://www.ciac.org/ciac/index.html• Value Added Services
– ICAT - http://icat.nist.gov/icat.cfm
© Open Security Foundation 2005
BIDBID
• Started in 1999, acquired by SecurityFocus on 07/17/2002
• Full time dedicated resources• Free, 72 hour delayed information
(SF researched)
© Open Security Foundation 2005
BID – Pros/ConsBID – Pros/Cons
• Pros– Brand awareness– Very detailed and technical information provided– Quick posting of new vulnerabilities due to
hosting of Bugtraq mail list
• Cons– Practices changed once acquired by corporation– Little response to feedback provided– Slow to load, banners ads a pain, 39 images per
entry– Product information based on erroneous
assumptions
© Open Security Foundation 2005
CVE/ICATCVE/ICAT
• MITRE and NIST• Full time dedicated resources,
federal funding• CVE started in 1999, ICAT ~2000• Both claim not to be a VDB• ICAT adds vulnerability classification
and statistics to a predominantly CVE based database
• Free
© Open Security Foundation 2005
CVE/ICAT – Pros/ConsCVE/ICAT – Pros/Cons
• Pros– Detailed statistics and classification scheme– Easy ability to download entire database– Widely adopted, heavily integrated into security
products
• Cons– Heavy use of CVE for vulnerability information– CVE “candidate” process slow and backlogged– Limited external references (ICAT)
© Open Security Foundation 2005
ISS X-ForceISS X-Force
• Run by Internet Security System (ISS)
• Full time resources dedicated• Started around Aug, 1997• VDB is free and public• Heavily used and referenced in ISS
security products• Fast and courteous reply to emails
with questions or errors
© Open Security Foundation 2005
ISS X-Force – Pros/ConsISS X-Force – Pros/Cons
• Pros– Very detailed, very thorough, historical entries– Fairly standard naming conventions– Very thorough external references
• Cons– Disclosure Issues– Many entries related to IDS events, not classic
vulnerabilities– No easy export, can’t easily integrate
© Open Security Foundation 2005
OSVDBOSVDB
• Open Security Foundation, 501(c)3 non-profit organization
• 3 project leaders, over 200 volunteers since inception
• First started on 08/30/2002• Free security information• Security community driven• Vendor dictionary, ethical disclosure
service, active integration
© Open Security Foundation 2005
OSVDB – Pros/ConsOSVDB – Pros/Cons
• Pros– Vendor Neutral, Un-biased– Integration with open source products– Broad source for data importation (sources,
dates)– Very thorough, attention to detail, historical
entries
• Cons– Slow updates on new vulnerabilities– Relies on community for resources– Currently no long term funding
© Open Security Foundation 2005
SecuniaSecunia
• Corporation located in Denmark• Full time staff• Launched 03/26/2003• Focus on timely vulnerability alerts• Free mailing list of new vulns
mailed daily
© Open Security Foundation 2005
Secunia – Pros/ConsSecunia – Pros/Cons
• Pros– Free mailing list– Very strong on monitoring vendor advisories and
updates– Attempt to work with open source community
• Cons– Lack of standards/confusing standards
• Issues lumped into “multiple” entries• Same vulnerability assigned a dozen entries, one per linux
vendor
– Only focuses on new vulnerabilities– Some solutions not practical or helpful
© Open Security Foundation 2005
Security TrackerSecurity Tracker
• Corporation in MD, USA• Full time resources dedicated• Started in 2002• Free weekly summary of
vulnerabilities, fee for instant alerts
© Open Security Foundation 2005
Security Tracker – Security Tracker – Pros/ConsPros/Cons
• Pros– Maintain their own standards, uniform
entries– Includes data source for vulnerability– Good data importation, monitor broad
source of information
• Cons– No statistics– Limited external references
© Open Security Foundation 2005
CERTCERT
• Carnegie Mellon, funded by US government
• Full time staff dedicated • Started in 1988, after Morris worm• Advisories for important issues• Maintains CERT-VU/KB Database• National Cyber Alert System
© Open Security Foundation 2005
CERT – Pros/ConsCERT – Pros/Cons
• Pros– US Federally funded and supported– Providing reports to technical and non-technical– Statistics provided
• Cons– Limited vulnerabilities tracked– Provide early information for exorbitant fee– Not always willing to coordinate with security
community– Serious questions about statistics, efficiency of
staff/funds– Overlap with CIAC and others
© Open Security Foundation 2005
CIACCIAC
• US funded and supported, DOE• Full time dedicated resources• Started in 1989• Advisories for major issues• Free service
© Open Security Foundation 2005
CIAC – Pros/ConsCIAC – Pros/Cons
• Pros– Stability, around since 1989– Updated regularly
• Cons– Limited vulnerabilities covered– Limited external references– Many advisories reprinted, no value added– Overlap with CERT
© Open Security Foundation 2005
Additional ResourcesAdditional Resources
• Vulnerability Sources Not Included:– COOP =
https://cirdb.cerias.purdue.edu/coopvdb/public/– Dragonsoft - http://vdb.dragonsoft.com/– FrSIRT -
http://www.frsirt.com/english/index.php– Securiteam - http://www.securiteam.com/– Sec Watch- http://www.secwatch.org/
• Focused Vulnerability Database– Nikto, Nessus– Sun, HP, IBM, Oracle, Microsoft, etc
• Vulnerability Sharing Clubs– http://www.idefense.com/– http://www.immunitysec.com
© Open Security Foundation 2005
Government FundedGovernment Funded
• CERT– The CERT/CC is funded primarily by the U.S. Department of Defense and
the Department of Homeland Security, along with a number of other federal civil agencies. Other funding comes from the private sector. As part of the Software Engineering Institute, we receive some funds from the primary sponsor of the SEI, the Office of the Under Secretary of Defense for Acquisition and Technology.
• CIAC– U.S. Department of Energy (DOE) funded
• CVE– CVE is sponsored by the National Cyber Security Division (NCSD) at the
U.S. Department of Homeland Security. US-CERT is the operational arm of the NCSD.
• ICAT– ICAT is maintained by the National Institute of Standards and
Technology.• US-CERT
– US-CERT is part of the Department of Homeland Security
• Little overlap? Consolidation? Oversight and audit?
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Research and Rankings
© Open Security Foundation 2005
Data HarvestingData Harvesting
• Where is information usually gathered?– Mail lists (Bugtraq, Full-disclosure, Vulnwatch,
Ntbugtraq)– Vendors (advisories)
• Where else should information be gathered?– Mail lists (Freshmeat, Vuln-dev, Dailydave, Pen-
test, other specialty security focused lists)– Vendors (Changelogs, Knowledge bases, Vendor
forums)– Exploit archives
© Open Security Foundation 2005
VDB IncestVDB Incest
• Who references who? Who refuses?– CVE: ISS, BID, Secunia, SecurityTracker, OSVDB– BID: CVE, Bugtraq, ISS, Secunia,
SecurityTracker, OSVDB– ISS: CVE, BID, Secunia, SecurityTracker, OSVDB– Secunia: CVE, OSVDB– SecurityTracker: CVE, OSVDB, Nessus– Nessus: CVE, BID, OSVDB– OSVDB: CVE, BID, Secunia, SecurityTracker,
ISS, Nessus, Snort, more
• Red denotes an apparent refusal to reference, even if the original point of disclosure or only available source.
© Open Security Foundation 2005
VDB RatingsVDB Ratings
• Based on important issues identified• Score of 1-10 provided for each of the
7 key performance areas• 1 = lowest, 10 = highest• Ratings given for each issue per VDB• Provides baseline for expectations for
each service• Identifies areas of improvements
© Open Security Foundation 2005
VDB Individual RankingsVDB Individual Rankings
• Ratings For Each Category• Top 3 VDBs• Top 3 Areas for VDB Improvement
See research posted at:• http://
www.opensecurityfoundation.org
© Open Security Foundation 2005
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Future
© Open Security Foundation 2005
Future of VDBsFuture of VDBs
• Long way to go• Hope to improve existing resources
– Better search interfaces– Better upkeep of older entries
• More services available to more people
• Further integration into products• Better statistics and trending
© Open Security Foundation 2005
Standardization of Standardization of DefinitionsDefinitions
• Risk ratings• Vulnerability Classifications
– Local vs. Remote (Remote Local)– Impact assessment (CIA)– Exploit availability– Access required to exploit
(Dependencies)
• Vulnerability definitions and terminology
© Open Security Foundation 2005
VDBs Suck - Expect MoreVDBs Suck - Expect More
• 20 years since inception, Limited improvements• Same mechanism for updating/verifying info• Very few classify or assign risk• Still no standardized classification for the few who do• Still no standardized risk value for the few who do • Still offer limited search ability overall • Many don't follow their own standards consistently• Most still very weak on external references • Barely any new services or ways to use information • Many don't seem to care about the vuln disclosure
process (why did it take 20 years for a vendor dict to emerge?)
• Bottom line, VDBs need to drastically improve
© Open Security Foundation 2005
Open Security Open Security FoundationFoundation
Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable
Brian Martin – jericho@attrition.orgJake Kouns – jkouns@opensecurityfoundation.org