Proposing SQL Statement Coverage Metrics
description
Transcript of Proposing SQL Statement Coverage Metrics
1
Proposing SQL Statement Coverage Metrics
Ben Smith
Yonghee Shin
Laurie Williams
2
Agenda
• Motivation and Objective
• Testing for Security
• SQL Injection Vulnerabilities/Attacks
• Terms and Metrics
• Case Study
• Limitations
• Future Work
3
Cyber Vulnerabilities (1995-2007)
0
1000
2000
3000
4000
5000
6000
7000
8000
1995 1997 1999 2001 2003 2005 2007
Year
Re
po
rte
d V
uln
era
bili
tie
s
4
Cyber Vulnerabilities, cont’d
• Averages from NVD 2003 – 2007 [1]
• Input validation vulnerabilities comprise >50% of reported total
• SQL Injection Vulnerabilities comprise 11% of reported total
5
Objective
To propose coverage metrics which determine the adequacy of a test set’s ability to mitigate SQL injection attacks.
6
Security Testing [2]
Intended Functionality (Requirements Specification)
Application as Coded
Unintended Functionality
Missing or Incorrect Functionality
7
$username = $_POST[‘username’];$password = $_POST[‘password’];
$result = mysql_query(“select * from users where username =‘’ OR 1=1 ---’ AND password = ‘$password’”);
$firstresult = mysql_fetch_array($result);
$role = $firstresult[‘role’];
$_COOKIE[‘userrole’] = $role
SQL Injection Attacks
$username = $_POST[‘username’];$password = $_POST[‘password’];
$result = mysql_query(“select * from users where username =‘$username’ AND password = ‘$password’”);
$firstresult = mysql_fetch_array($result);
$role = $firstresult[‘role’];
$_COOKIE[‘userrole’] = $role;
‘ OR 1=1 --
8
Web Application Security
InternetInternetData
Web App
Query
Query
9
Coverage Analysis
10
Theory
Higher SQL Coverage Metrics Higher System
Security
11
Research Plan
Blacklist/Input Validation Testing
SQL Coverage Metrics
12
Terms and Metrics (1)
Target Statement: a SQL statement which could cause a security problem when malicious input is used, for example:
java.sql.Connection conn = factory.getConnection();java.sql.PreparedStatement ps =
conn.prepareStatement("UPDATE globalVariables SET value = ? WHERE name = ?;");
ps.setInt(1, value);ps.setString(2, name);java.sql.ResultSet rs = ps.executeQuery();
13
Terms and Metrics (2)
Input Variable: any variable in the server-side production code which is dynamically user-assigned and sent to the database management system. For example,
java.sql.Connection conn = factory.getConnection();java.sql.PreparedStatement ps =
conn.prepareStatement("UPDATE globalVariables SET value = ? WHERE name = ?;");
ps.setInt(1, value);ps.setString(2, name);java.sql.ResultSet rs = ps.executeQuery();
14
Terms and Metrics (3)
Target Statement Coverage: the percentage of target statements in the production code executed/covered at least once by the test set.
Input Variable Coverage: the percentage of input variables in the production code executed/covered at least once by the test set.
15
Case Study: iTrust [3]
• Role-based web healthcare application
• Developed by NC State students in a security testing class
• Aims to be HIPAA-compliant
• ~7700 SLOC, 143 production classes
• Tomcat + MySQL• 84% line coverage
16
Case Study: Overview
• Manually instrument iTrust to record the first execution of each target statement
• Run the built-in unit tests to reveal the coverage result for both SQL statement coverage and input variable coverage
• Warning: Formative Research!!
17
Case Study: Results
Coverage Metric Tested Total Result
Target Statement 90 93 96.7%
Input Variable 209 212 98.5%
18
A few more results…
• Three-tiered web applications
• SQL
• Tomcat
• Provided test sets
• Examples– Hispacta– IceScrum– Care2x?
19
A few more results…
iTrust IceScrum Hispacta
Type of Tests Run Unit Unit Unit + Integration
Lines of Code 7,707 19,442 1,991
Production Classes 143 155 42
DAO Classes 20 10 4
SQL Statements 93 96 23
Covered by Tests 90 29 18
Coverage 96.7% 30.2% 78.3%
SQL Input Variables 212 517 36
Covered by Tests 209 258 32
Coverage 98.5% 49.9% 88.9%
Line Coverage (EclEmma)
84.1% 9.7% 49.4%
20
Threats to Validity
• Human error
• Inaccurate analysis
• May not be feasible in other systems, languages or operational environments
• iTrust is highly testable, and well-tested
21
Future Work
• Empirical studies– SQL coverage in other open source systems– Linking SQL coverage to higher security levels
• Automation– Dynamic Queries– Identification
• Finer-grained metrics
• Use as platform for blacklist/whitelist testing
22
Very Future Work
Blacklist/Input Validation Testing
SQL Coverage Metrics
A Framework
23
References
[1] United States National Vulnerability Database. http://nvd.nist.gov
[2] H. H. Thompson, "Why security testing is hard," Security & Privacy Magazine, IEEE, vol. 1, no. 4, pp. 83-86, 2003.
[3] iTrust website. http://sourceforge.net/projects/iTrust
24
•Questions?
•Feedback & Comments Welcome!
25
Dynamic Queriesjava.sql.Connection conn = factory.getConnection();java.sql.PreparedStatement ps = null;
if (user.isAdmin()){
ps = conn.prepareStatement(“select * from adminTable;");
}else{
ps = conn.prepareStatement(“select * from generalTable;”);
}
ps.executeQuery();
26
Case Study: Instrumentation
java.sql.Connection conn =factory.getConnection();java.sql.PreparedStatement ps =conn.prepareStatement("UPDATEglobalVariables set SET VALUE = ? WHEREName = ‘Timeout’;");ps.setInt(1, mins);SQLMarker.mark(1, 1); java.sql.ResultSet rs = ps.executeQuery();