Composing a Framework to Automate Testing of
Operational Web-Based Software
Reham Alhejiali, Chris Cederstrom, Ranjitha Kashyap
Motivation
Web based application:Example: E-commerceCharacteristic:1.Short time to market.2.Extensively used.3.High reliability expected.4.Demand for continuous availability.5.Large user community.
Goal
-To address the need for an automated cost- effective testing strategy for web application.
- To investigate the scalability and evolving the tests suites to automatically adapt as the application operation profile changes.
Outline
1.General framework for testing web-based application.
2. State of the art.
3.The testing framework .
4. Demonstrates the use of the framework with case study.
5. conclusion.
State of the Art
• User-Session-based testing
-Test cases: - Created by capturing real user interactions and utilizing the user sessions as
representative test cases of user behavior. -Testers could use the collected user sessions during maintenance to enhance the
original test suite. -The user sessions provide the test data that represents usage not anticipated during
earlier testing stages and that evolves as operational profiles change.
-Webking and Rational Robot
Generated test cases may not be adequate.
-Elbaum et al(2003)
Reduction technique employed is not scalable.
continue:
• Functional testing:
-link, form testers
-Test the functionality of application.
• Program-based testing
-Test cases: based on the data flow between objects in the model.
Liu et al (2000)
-WATM
-Test based on the data flow between objects in the model.
-ICFG
Di Lucca et al(2002)
-Developed a web application model and set of tools for the evaluation and automation of testing web application.
-Test case generation not completely automated.
Test Case Generation
• Using a Web server to log user sessions for test case generation
• Modified the AccessLog class of the Resin web server to log user sessions in a specified format:o IP addresso Timestampo Requested URL o Cookieso Name-value of GET/POST requestso Referrer URL
• Each user session is a collection of user requests in the form of URL and name-value pairs.
Replay Tool & Coverage Analysis
• Replaying user sessions are viewed using wget for each entry in the user session log
• These requests will emulate user interactions and input
• Cookies are used to recognize the user and sessions
• Coverage analysis is to evaluate the effectiveness of the generated test suite
• Coverage is calculated by Clover
• Clover returns the analysis of statements and methods used
Oracle and Fault Detection
• Oracle generate expected results using a properly working application
• Run the user sessions on the working application and new version and store results in database
• Results of working application is compared to output from test runs to confirm validity and detect faults
Case Study
• A medium sized web application like bookstore was considered for test
• Front end had 11 jsps and classes
• LOC:9,748, Methods: 385
• Collected 123 User sessions
WEBAccessLog
User
User Session Log
0.197.37.159 [03/Feb/2004:16:14:05 −0500] GET /apps/bookstore/Default.jsp ] −−cookies=off] −
10.197.37.159 [03/Feb/2004:16:16:27 −0500] GET /apps/bookstore/Registration.jsp ] −−cookies=off −−header "Cookie:JSESSIONID=a7mpavbwGTf6"] http://dwalin.cis.udel.edu:8080/apps/bookstore/Default.jsp
10.197.37.159 [03/Feb/2004:16:17:22 −0500] GET /apps/bookstore/Registration.jsp
?member_login=bobmason& member_password=14921492&member_password2=14921492& first_name=bob&last_name=mason& email=bobmason%40udel.edu&address=&phone=& card_type_id=&card_number=& FormName=Reg&FormAction=insert& member_id=&PK_member_id= ]
−−cookies=off −−header "Cookie:JSESSIONID=a7mpavbwGTf6"] http://dwalin.cis.udel.edu:8080/apps/bookstore/Registration.jsp
Test case generation and execution
AccessLog
Access Log Parser
10.197.37.159: GET.apps.bookstore.Default.jsp GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Default.jsp GET.apps.bookstore.Login.jsp POST.apps.bookstore.Login.jsp
;
10.82.161.133: GET.apps.bookstore.Default.jsp GET.apps.bookstore.images.icon_reg.gif GET.apps.bookstore.images.icon_home.gif GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Registration.jsp GET.apps.bookstore.Default.jsp
;
Concept Analysis Tool
objects[001]: 0.82.161.133
attributes[001]: GET.apps.bookstore.images.icon_home.gif
GET.apps.bookstore.images.icon_reg.gif
Test Suite Reducer
Replay Tool
Seeded Fault- Technique to measure Test Coverage
- Known bugs are randomly added to the source code
- During testing, the percentage of known bugs not found indicates the real bugs that remain.
Fault Description Example
Logic error in control or data flow //String bookDir = bookProp.getProperty(title)bookDir = null;
Link error in hyperlink location String link1 = "registration.jsp"String link2 = "shoppingCart.jsp"<a href="<%=link2>">Registration</a>
Datastore error in code that interacts with data store
/* Select book_id, name, quantity from store where book_id=123 */Select title, name, quantity from store where book_id=123
Cost and Scalability
Component Execution Time Component Output Output Space
Test Case Generation 19s Reduced Test Suite 1MB
Replay Tool Execution
16m56s Pages Retrieved 152 MB
Coverage Analysis 1m52s Coverage Reports/Clover Database
3.5MB
Test Oracle 14s Diff Output 0.5M
Top Related