Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong...
-
Upload
ella-higgins -
Category
Documents
-
view
215 -
download
0
Transcript of Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong...
![Page 1: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/1.jpg)
Formal Testing of Web Content using TTCN-3
By Robert Probert, Bernard Stepien, Pulei Xiong
University of Ottawa
![Page 2: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/2.jpg)
Motivation
• Web Applications (WA) are the most widespread target domain of software development in computer history.
• WA are error prone because mostly because anyone can develop them and they are mostly distributed.
• Testing of WAs: manual or using dedicated tools or robots.
![Page 3: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/3.jpg)
Current state of researchon formal Web content testing
• XML based formalism
• Xiaoping Jia and Hongmin Liu, Rigorous and Automatic Testing of Web Applications
• Sahil Thaker, Robust Testing for Web Applications
![Page 4: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/4.jpg)
Web content testing elements
• Hyperlink checking, including recursive• Text fragment checking• Protocol checking• Table checking• Form checking• Formatting checking• Mirror sites content checking• Version checking (languages,etc…)• Invalid HTML syntax checking• …
![Page 5: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/5.jpg)
Specific needs of web content testing
• TTCN has been originally designed for telecommunication protocols.
• In Telecom, every single data element is relevant and needs to be coded and decoded to and from the abstract layer.
• In Web application, only part of the content is testable and needs to be extracted because most of the web page content is either too large or unpredictable.
• Thus, in web content testing, we test only invariants.
![Page 6: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/6.jpg)
DePaul U Formalism BNFTestSpec ::= TestSuite*TestSuite ::= TestCase*TestCase ::= TestStep +TestStep ::= Name Condition opt RequestSpec
ResponseSpec TestStep*RequestSpec ::= URL RecursiveSpec opt
HearderSpec* ParameterSpec*VariableDecl*
ResponseSpec ::= StatusCode ContentTypeHearderSpec*VariableDecl* Predicate +
RecursiveSpec ::= Depth Domain optHeaderSpec ::= Name CDATA optional optParameterSpec ::= Name CDATA optional optCondition ::= PredicatePredicate ::= not Predicate | Predicate and predicate | Predicate or Predicate
| Predicate implies Predicate | ( forall VariableDecl + Predicate )| ( exists VariableDecl + Predicate ) | SimplePredicate
SimplePredicate ::= MatchPred | ContainPred | ComparePredMatchPred ::= SelectExp MatchOp ( RegExp | CDATA )MatchOp ::= equals | contains | startsWith | endsWithComparisonPred ::= Exp ComparisonOp ExpExp ::= NumberExp | DateExpComparisonOp ::= == | != | < | <= | > | >=ContainPred ::= HrefSpec | FormSpecHrefSpec ::= URL CDATA opt VariableDecl optFormSpec ::= ActionSpec opt MethodSpec optVariableDecl ::= Name SelectExp
![Page 7: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/7.jpg)
Text Fragment checkingDePaul U formalism
<testcase name="Content check using predicates"> <teststep name="Content check step one"> <request url="http://www.cs.depaul.edu/program"/> <response statuscode="200"> <and>
<or> <match op="contains" regexp="false" select="/html/body"
value="Undergraduate Degree"/> <match op="contains" regexp="false" select="/html/body"
value="Bachelor Degree"/></or><match op="contains" regexp="true" select="/html/body”
value="[M|m]aster [D|d]egree"/> </and> </response> </teststep></testcase>
![Page 8: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/8.jpg)
Mapping DePaul U Formalism to TTCN-3
• Testsuite -> testsuite• Testcase -> testcase• Teststep -> function• Request -> send• Response -> receive• Match -> built-in TTCN-3 matching mech.• Regexp -> pattern• Value -> template
![Page 9: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/9.jpg)
Text Fragment checking TTCN-3types and templates
template charstring program_page_web_url := "http://www.cs.depaul.edu/program";
type record content_response_type {charstring status,charstring content
}
template content_response_type content_response_pred := {status := "HTTP/1.1 200 OK",content := pattern "*<BODY
*[Undergraduate Degree | Bachelor Degree]*[[M|m]aster [D|d]egree]*</BODY>*"
}
![Page 10: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/10.jpg)
Text Fragment checking TTCN-3test case
testcase Content_check_pred_TC() runs on MTCType system SystemType {var verdicttype theOverallVerdict;
map(mtc:web_port, system:system_web_port);
theOverallVerdict := CheckContent_pred(program_page_web_url, content_response_pred);
setverdict(theOverallVerdict);}
![Page 11: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/11.jpg)
Text Fragment checking TTCN-3test logic
function CheckContent_pred(charstring theURLLink, content_response_type theContent_response) runs on MTCType return verdicttype {
var charstring theBadResponse;var verdicttype theFinalVerdict;
web_port.send(theURLLink);alt {
[] web_port.receive(theContent_response) {theFinalVerdict := pass;
}[] web_port.receive(charstring:?) -> value theBadResponse {
log("Bad Response: " & theBadResponse);theFinalVerdict := fail;
}}return theFinalVerdict;
}
![Page 12: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/12.jpg)
Contains concept and TTCN-3 pattern
• The ‘and’ operator does not imply sequence.
• Permutations of values must be specified in the TTCN-3 template using pattern.
• Combinatorics explosion when the number of text fragments increases (np).
• TTCN-3 should implement a concept of “contains” with logical expressions
![Page 13: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/13.jpg)
Evaluation of TTCN-3 advantages with text fragment checking
• DePaul U has the contains concept.
• TTCN-3 has better structuring of data sent or received.
• TTCN-3 has the concept of alternative to structure exceptions, etc…
• TTCN-3 is more flexible because new functionalities can be added both at the abstract and adapter level.
![Page 14: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/14.jpg)
Recursive hyperlink checkingDePaul formalism
<testcase name="Link check"><teststep name="link check 1">
<request url="http://www.cs.depaul.edu", recursive="true", recursivedepth="3"/>
<response statuscode="200"/></teststep>
</testcase>
![Page 15: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/15.jpg)
Recursive hyperlink checkingTTCN-3
testcase Link_Check_TC() runs on MTCType system SystemType {...theOverallVerdict := CheckChildLink(main_page_web_url, 1);setverdict(theOverallVerdict);
}
Solution: Test case invokes a recursive function and appropriate list type definitions.
type record of charstring HrefListType;type record hyperlink_response_type {
charstring status,HrefListType hrefList
}template hyperlink_response_type hyperlink_response :={
status := "HTTP/1.1 200 OK",hrefList := ?
}
![Page 16: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/16.jpg)
Recursive hyperlink checkingTTCN-3 recursive function
function CheckChildLink(charstring theURLLink, integer theDepth) runs on MTCType return verdicttype {
web_port.send(theURLLink); web_port.receive(hyperlink_response) -> value theHyperlink_response;
if(theDepth <= maxDepth) {
var integer theNewDepth;var HrefListType theResponseHrefList :=theHyperlink_response.hrefList;var integer numOfLinks := sizeof(theResponseHrefList);var integer i;theNewDepth := theDepth + 1;for(i:=0; i < numOfLinks; i:=i+1){
oneVerdict := CheckChildLink(theResponseHrefList[i], theNewDepth);if(oneVerdict == fail) { theFinalVerdict := fail; }
} } return theFinalVerdict; }
![Page 17: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/17.jpg)
Evaluation of TTCN-3 advantages with recursive link checking
• TTCN-3 is more powerful because it allows more filtering on the recursion. For example, one would only recurse through the links to other web pages internal to the same company and ignore external links.
• TTCN-3 allows more sophisticated recursion control than simple depth limits.
• DePaul formalism is more compact but also more limited.
![Page 18: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/18.jpg)
Quantified predicate checkingDePaul U Formalism
<testsuite> <testcase name="Content check using quantified predicates"> <teststep name="content check step one" > <request url="http://jordan.cs.depaul.edu/webtest/testhref.htm"> </request> <response>
<forall><varbale name="l" select="descendent::a/@href"/><match op="startswith" select="$l" regexp="false"
value="http://"/></forall>
</response> </teststep> </testcase></testsuite>
![Page 19: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/19.jpg)
Mapping to TTCN-3
• Forall -> TTCN-3 for loop with bounds
• StartsWith -> TTCN-3 substring
• Select -> done in TTCN-3 CODEC
![Page 20: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/20.jpg)
Quantified predicate checkingTTCN-3
testcase Forall_protocol_check_TC(…) runs on MTCType system SystemType {
web_port.send(theURLLink); web_port.receive(hyperlink_response) -> value theHyperlink_response;
all_match := true; for(i:=0; i < numOfLinks; i:=i+1) {
var HrefListType theResponseHrefList := theHyperlink_response.hrefList; if(substr(theResponseHrefList[i],0,theLength) != theProtocol) {
all_match := false;log("link: " & theResponseHrefList[i] &
" is not of protocol " & theProtocol);}
} if(all_match) { setverdict(pass); } else { setverdict(fail); } }
![Page 21: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/21.jpg)
Remarks about the TTCN-3 solution
• Matching is not achieved via the inherent TTCN-3 matching mechanism, but instead using traditional programming languages features (substr). This is mainly because it is impossible to match a list of unknown length.
• There is no parametric solution. Two different select values would require a dedicated definition of types, templates and test cases for each select values.
![Page 22: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/22.jpg)
Our Recommendationparametric CoDec
type record myRecord {integer field_1,charstring field_2 codecparameter, charstring field_3
}
template myRecord myTemplate := {field_1 := 5, field_2 := “href”, field_3 := “abcd”
}
Some fields act as parameters passed to the CODEC rather than matching values coming from the CODEC
![Page 23: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/23.jpg)
Sahil Thaker WebDeveloppers ltdForm testing formalism
<oracle scope = form><form method=“get” target=“”><input type=hidden name=“” value=“”><input type=submit>
</oracle>
![Page 24: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/24.jpg)
Form testing in TTCN-3type definitionstype record FormResponseType {
charstring status,FormType form
}type record FormType {
charstring name,charstring method,charstring action,InputListType inputList
}type record InputType {
charstring name,charstring type_input,integer size_input,charstring value_input
}type record of InputType InputListType ;
![Page 25: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/25.jpg)
Form testing in TTCN-3template definitions
template FormType theRegistrationForm_template := { name := "subscription_form", method := "POST", action := "http://www.someone.com/cgi-bin/processSubscription.pl", inputList := { {name :="firstname", type_input := "text", size_input := 20, value_input := ""} , {name :="lastname", type_input := "text", size_input := 35, value_input := ""} , {name :="city", type_input := "text", size_input := 40, value_input := ""}, {name :="send_me_emails", type_input := "checkbox",
size_input := 0, value_input := ""} , {name :="wine_and_cheese", type_input := "checkbox",
size_input := 0, value_input := ""} , {name :="", type_input := "submit", size_input := 0, value_input := "send"} , {name :="", type_input := "reset", size_input := 0,
value_input := "Oooops!_Let_me_try_again!"} }}
![Page 26: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/26.jpg)
Form testing in TTCN-3parametric testcase definition
testcase Form_Check_TC(charstring theURLLink, FormResponseType theFormResponse) runs on MTCType system SystemType {
…
web_port.send(theURLLink);alt {
[] web_port.receive(theFormResponse) -> value theFormResponseTypeValue{
log("the form " & theFormResponseTypeValue.form.name & " conforms to specification");
setverdict(pass);}[] web_port.receive(FormResponseType:?) {
log("in the catch all receive !");setverdict(fail);
}}
}
![Page 27: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/27.jpg)
Form testing in TTCN-3test control definition
control {execute (Form_Check_TC(form_page_web_url,
form_response(theRegistrationForm_template)));execute (Form_Check_TC(another_web_page_url,
form_response(another_form_template)))}
Maximizing re-usability using:•Parametric test cases•Parametric data templates
![Page 28: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/28.jpg)
Advantages of TTCN-3
• Parametric test cases
• Parametric templates
• Structuring at all levels, types, templates, test cases, functions
• Concept of control
• Test configuration specification
![Page 29: Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa.](https://reader030.fdocuments.us/reader030/viewer/2022032717/56649e0c5503460f94af4923/html5/thumbnails/29.jpg)
Conclusions
• TTCN-3 is adequate for web content testing.
• There is no need to create new formalism dedicated to web content testing.
• Instead, it would be more beneficial to use TTCN-3 to develop generic web content testing tools mostly because of TTCN-3 parametric features.