Web Bypass Testing

An Industrial Case Study of Bypass Testing on Web Applications

Joint research Dr. Joint research Dr. Ye WuYe Wu, , Xiaochen DuXiaochen Du, , Hong HuangHong Huang, , Vasileios PapadimitriouVasileios Papadimitriou, , Qingxiang Wang Qingxiang Wang and and Joann J. OrdilleJoann J. Ordille of Avaya Labs Research of Avaya Labs Research

Based on papers in ISSRE 2004 and ICST 2008Based on papers in ISSRE 2004 and ICST 2008

Jeff OffuttJeff OffuttSoftware EngineeringSoftware Engineering

George Mason UniversityGeorge Mason University

Fairfax, VA USAFairfax, VA USA

www.cs.gmu.edu/~offutt/www.cs.gmu.edu/~offutt/

[email protected]@gmu.edu

Expanded version of a talk given at the first Expanded version of a talk given at the first International Conference on Software Testing, International Conference on Software Testing,

Verification and ValidationVerification and Validation

Outline of Talk

•Motivation•Applying bypass testing –

early results•Automating bypass testing•Real-world examples•Industrial case study•Conclusions and future work

ICST 2008 © Jeff Offutt 2

Web Application Input Validation

Sensitive Data

Bad Data• Corrupts data base• Crashes server• Security violations

Check data

Check data

Malicious Data

Can “bypass” data checking

Client

Server

© Jeff Offutt, 2004 4

Deploying Software

• Bundled : Pre-installed on computer

• Shrink-wrap : Bought and installed by end-users

• Contract : Purchaser pays developer to develop and install, usually for a fixed price

• Embedded : Installed on a hardware device, usually with no direct communication with user

• Web : Executed across the Internet through HTTP

© Jeff Offutt 5

Problem Parameters• HTTP is a stateless protocol

– Each request is independent of previous request

• Servers have little information about where a request comes from

• Web site software is extremely loosely coupled– Coupled through the Internet – separated by space– Coupled to diverse hardware devices– Written in diverse software languages

ICST 2008

© Jeff Offutt 6

Bypass Testing

• “bypass” client-side constraint enforcement

• Bypass testing constructs tests to intentionally violate constraints :– Eases test automation– Validates input validation– Checks robustness– Evaluates security

ICST 2008

© Jeff Offutt 7

User Name:

Small

$150

Version to purchase:

Age:

Large

$500

Medium

$250

Simple Example Web Page

ICST 2008

© Jeff Offutt 8

User Name:

Small

$150

Version to purchase:

Age:

Large

$500

Medium

$250

Username should be plain text only.

Age should be between 18 and 150.

Invalid data, please correct …

Alan<Turing 500

Proper Behavior

ICST 2008

© Jeff Offutt 9

Abbreviated HTML<FORM >

<INPUT Type=“text” Name=“username” Size=20>

<INPUT Type=“text” Name=“age” Size=3 Maxlength=3>

<P> Version to purchase:

…

<INPUT Type=“radio” Name=“version” Value=“150” Checked>

<INPUT Type=“radio” Name=“version” Value=“250”>


<INPUT Type="submit" onClick="return checkInfo(this.form)">

<INPUT Type=“hidden” isLoggedIn=“no”>

</FORM>ICST 2008

© Jeff Offutt 10

Bypass Behavior

• Extremely loose coupling …

• combined with the stateless protocol …

• allows users to easily bypass client-side checking :

Users can Users can savesave and and modifymodify the HTML the HTML

ICST 2008

© Jeff Offutt 11

Saved & Modified HTML<FORM >

<INPUT Type=“text” Name=“username” Size=20>

<INPUT Type=“text” Name=“age” Size=3 Maxlength=3>

<P> Version to purchase:

…



<INPUT Type=“radio” Name=“version” Value=“500” Checked>

<INPUT Type="submit" onClick="return checkInfo(this.form)">

<INPUT Type=“hidden” isLoggedIn= “no” >

</FORM>

Allows an input with arbitrary age, no checking, cost=$25 …

‘<‘ can crash an XML parser

Text fields can have SQL statements

25

yes

ICST 2008


SQL Injection

User Name: Password:turing enigma

Original SQL:

SELECT username FROM adminuser WHERE username='turing' AND password ='enigma'

“injected” SQL:

SELECT username FROM adminuser WHERE username='turing‘ OR ‘1’ = ‘1’ AND password ='enigma‘ OR ‘1’ = ‘1’

’ OR ‘1’=‘1 ’ OR ‘1’=‘1

© Jeff Offutt 13

Applying Bypass TestingApplying Bypass Testing

• Analyze HTML to extract each form element

• Model constraints imposed by HTML and JavaScript

• Rules for data generation :– From client-side constraints– Typical security violations– Common input mistakes

ICST 2008

Validating input data on the client is like asking Validating input data on the client is like asking your opponent to hold your shield in a sword fightyour opponent to hold your shield in a sword fight


Example Client-Side Constraint Rules

• Violate size restrictions on strings

• Introduce values not included in static choices– Radio boxes– Select (drop-down) lists

• Violate hard-coded values

• Use values that JavaScripts flag as errors

• Change “transfer mode” (get, post, …)

• Change destination URLs


Example Server-Side Constraint Rules

• Data type conversion

• Data format validation

• Inter-field constraint validation

• Inter-request data fields (cookies, hidden)


Example Security Violation Rules

Potential Illegal Character Symbol

Empty String

Commas ,

Single and double quotes ’ or ”

Tag symbols Tag symbols < and >

Directory paths .. ../

Strings starting with forward slash /

Strings starting with a period .

Ampersands &

Control character NIL, newline

Characters with high bit set 254 and 255

Script symbols <javascript> or <vbscript>


First Example CyberChair

• CyberChair : Web-based conference management– www.cyberchair.org

– ICSE , ICST, ISSRE, ICSM, …

• Bypass testing found 5 types of faults1. Submission without authentication

2. Unsafe use of hidden form field

3. Disclosing information (program crashes)

4. Lack of validation of file type

5. Allows papers of negative length

Automating Bypass Testing• Autobypass : A web application that accepts a URL

and generates input data for the HTML form fields– Also accepts any needed login data– MS thesis by Vasileios Papadimitriou

• Built on top of HttpUnit– Parses HMTL pages– Identifies forms and their fields – Creates bypass test cases – Submits test cases to the application’s server


Types of Client Input Validation• Client side input validation is performed by HTML form

controls, their attributes, and client side scripts that access DOM

• Validation types are categorized as HTML and scripting– HTML supports syntactic validation– Client scripting can perform both syntactic and semantic validation


HTML Constraints Scripting Constraints• Length (max input characters)

• Value (preset values)

• Transfer Mode (GET or POST)

• Field Element (preset fields)

• Target URL (links with values)

• Data Type (e.g. integer check)

• Data Format (e.g. ZIP code format)

• Data Value (e.g. age value range)

• Inter-Value (e.g. credit # + exp. date)

• Invalid Characters (e.g. <,../,&)

Example Interface:yahoo registration form


Limited Length (HTML)

Preset Values (HTML)Preset Transfer Mode in form definition (HTML)

Preset No of Fields (HTML)

URL with preset Values (HTML)

Data Value, Type, & Formatvalidation (script)

Inter Value validation (script)

http://us.rd.yahoo.com/reg/img_help/us/*https://edit.yahoo.com/config/form?.branch=&.form=secimg_help&.intl=us

Test Value Selection• Challenge:

– How to automatically provide effective test values?• Semantic Domain Problem (SDP)

– Values within the application domain are needed – Enumeration of all possible test values is inefficient

• Possible Solutions– Random Values (ineffective – lots of junk)– Automatically generated values (very hard)– Taking values from session log files (feasible but incomplete)– Tester input (feasible)

• AutoBypass uses an input domain created by parsing the interface and tester input


AutoBypass

• AutoBypass steps (the big picture)


Parse Interface

Set Default Values

Generate Test Cases& Run Tests

Review Results

• All HTML violation rules are used to generate test cases

• First version of AutoBypass does NOT automatically violate scripting validation, but :– AutoBypass behaves as a browser with scripts disabled

– Tester can provide test inputs that will bypass scripting validation.

Real-World Examples


atutor.caAtalker

demo.joomla.orPoll, Users

phpMyAdmin Main page, Set Theme, SQL Query, DB Stats

brainbench.comSubmit Request Info, New user

myspace.comEvents & Music Search

bankofamerica.comATM locator, Site search

comcast.com Service availability

ecost.com Detail submit, Shopping cart control

google.com Froogle, Language tools

pageflakes.comRegistration

wellsfargolife.com Quote search

nytimes.comUs-markets

mutex.gmu.eduLogin form

yahoo.com Notepad, Composer, Search reminder, Weather Search

barnesandnoble.comCart manager, Book search/results

amazon.com Item dispatch,Handle buy

Pure black-box testingPure black-box testingmeansmeans

no source (or permission) no source (or permission) needed !needed !

Classifying Output Responses• (V) Valid Responses : invalid inputs are adequately processed by the

server

• (F) Faults & Failures : invalid inputs that cause abnormal server behavior (typically caught by web server when application fails to handle the error)

• (E) Exposure : invalid input is not recognized by the server and abnormal software behavior is exposed to the users


(V1) Server acknowledges the invalid request and provides an explicit message regarding the violation

(V2) Server produces a generic error message

(V3) Server apparently ignores the invalid request and produces an appropriate response

(V4) Server apparently ignores the request completely

Results


v


““Knowing is not enough, we must apply. Willing is not enough, Knowing is not enough, we must apply. Willing is not enough, we must do.” Goethewe must do.” Goethe

Research to PracticeThey’re teaching a new way of plowing over at the Grange tonight - you going?

Naw - I already don’t plow as good as I know how...

Industrial Case Study

• Inventions from scientists are slow to move into industrial practice

• We wanted to investigate whether the obstacles are :– Technical difficulties of applying to industrial use– Social barriers– Business constraints

• Tried to technology transition bypass testing to the research arm of a software company


Avaya’s NPP Technology• Avaya Labs Research creates research prototypes of

software systems, then turns successful prototypes over to product groups

• NPP : Notification Preference Portal– Users specify how and when they should be contacted– Types include phone, email and SMS– Contacts an be made in parallel or sequentially

• Used to notify users of events

• This study was part of system testing– NPP is now in production


NPP Design and Implementation• NPP is a highly user interactive web application

• Uses many screens

• Javascript is used on the client to :– Validate inputs– Dynamically modify screen by manipulating DOM– Encode input data into XML before sending to the server

• The second two uses necessitated changes to how bypass testing was applied


NPP Bypass Tests• Bypass testing analyzes HTML (statically) and

generates inputs that violate input constraints

• The extensive modification of the HTML DOM meant HTML could not be analyzed statically

• Instead, the input requirements of the server software were identified, and tests encoded in XML– A special-purpose tool was written to convert XML tests

into HtmlUnit tests


NPP Testing Results• Six NPP screens were tested

• Tests are invalid inputs – exceptions are expected

• Effects on back-end were not checked– Failure analysis just based on response screens


Web Screen Tests Failing Tests Unique Failures

Points of Contact 42 23 12

Time Profile 53 23 23

Notification Profile 34 12 6

Notification Filter 26 16 7

Change PIN 5 1 1

Create Account 24 17 14

TOTAL 184 92 63

Types of Faults• Invalid data saved into the database

– These lead to more visible failures later– Example : An invalid password was accepted, but the

account could not subsequently be used– Example : Subsequent messages could not be sent to

invalid contacts

• No response at all– Probably a software component failed– Database or web server sometimes crashed

• Exposure errors– Internal exception message sent in the response screen


Conclusions• Bypass testing worked very well in an industrial

context– There is no technical obstacle to adoption– Source is not needed

• Even hand generation of tests was quite cheap in comparison with other methods– There is no valid business barrier

• Most problems are unlikely with non-malicious users and a correct implementation– But client-side validation is notoriously error-prone


We conclude the primary obstacle is socialWe conclude the primary obstacle is social

Future Work• A major observability problem with web application testing is

detecting invalid database values– A comprehensive valid data model could allow database auditors to

be developed

• Javascript needs to be fully parsed and analyzed– Implement scripting violation rules

• Widen the scope of testing from a form/ to a site– Test sequence of events – Application level Input Domain

• Explore possibilities for automated response evaluation

• Ajax allows client-server messages to be sent asynchronously through message passing– This introduces more controllability and observability problems


Web Bypass Testing

Documents

Transcript of Web Bypass Testing