Evaluating Web Evaluating Web Software ReliabilitySoftware Reliability

By Zumrut Akcam, Kim Gero, Allen Chestoski,Javian Li & Rohan Warkad

CSI518 – Group 1CSI518 – Group 1

Agile DevelopmentAgile Development

The Agile Development ProcessThe Agile Development Process

How does Agile work?

How did our class use Agile?

3 Sprints

“Stand up” meetings at beginning of each class

Retrospective at the end of each sprint

OverviewOverview

Definition of ReliabilityDefinition of Reliability

What is reliability for Web applications?

The reliability for Web applications can be defined as the probability of failure-free Web operation completions.[1]

Failure is “the event of a system deviating from its specified behavior like obtaining or delivering information”.[2]

Failure SourcesFailure Sources

Failures are caused from the following sources:

Host, network or browser failures: computer systems, network or software failures, etc.

Source content failures: missing, inaccessible files, JavaScript errors, etc.

User errors: improper usage, mistyped URLs.[1]

Project GoalProject Goal

To strengthen the reliability of Web applications by minimizing the number of source content failures.

Attempt to extend work on testing the reliability of websites.

Gain experience doing a research project

Sprint 1Sprint 1

Sprint 1 GoalsSprint 1 Goals

Read relevant research papers

Identify factors that may effect reliability analysis

Determine a system to analyze reliability on

Gather access and error logs

Factors That May EffectFactors That May EffectReliability AnalysisReliability Analysis

Byte Count

User Count

Session Count

Error Count

System to Analyze Reliability OnSystem to Analyze Reliability On

Reliability analysis via error logs

Variety of reliability requirements

Commercial and non-commercial

We will try to record the technologies the websites employ (Apache, DNN, ISS, PHP, ColdFusion, etc..)

Sprint 2Sprint 2

Sprint 2 GoalsSprint 2 Goals

Collect log files for calculation

Automate processes to extra data (user, session, byte, and error counts)

Convert them into excel format

Log Parser

Sprint 2 ProgressSprint 2 Progress

DNN Logs (10 Websites)

PHP Logs

What is DotNetNuke (DNN)What is DotNetNuke (DNN)

.NET version of Drupal An open source platform for building websites and web applications based on Microsoft .Net technology. Leading open source ASP.NET web content management Has been downloaded over 6 million times ~100 employees 5th Version Founded 2006

Our DNN LogsOur DNN Logs

Logs from 10 Websites Window Server (Same Server) SQL Server 2008 ~1000 unique visitors per day Logs contain

User count Limited Error count

Major ProblemMajor Problem

Our DNN Logs does not contain Session count

Byte count

AlternativeAlternative

Generate our own DNN logs

Sprint 3Sprint 3

Server SideServer Side

Technologies Used Windows XP Professional Microsoft Internet Information Servers (ISS) Microsoft SQL Server 2008 DotNetNuke (DNN)

Logs Generated Client IP’s Byte Counts (Uploaded & Downloaded) Time-Taken Status Code

Generating LogsGenerating Logs

Clients

Web-Crawlers

DotNetNuke Client API

Inducing Errors

ResultsResults

Workload Measurement FactsWorkload Measurement Facts

Server log data consisted of 23 consecutive days of data.

Page Not Found (Error 404) is the most common type of error in our logs, with 46% of total recorded errors.

Accessing forbidden data (Error 403) follows with 41%.

72 unique IPs, 32970 hits total, and each hit associated with average 5020 bytes.

Error/Success RatiosError/Success RatiosHTTPStatus Codes

Description

200 OK

206 Partial Content

302 Found

304 Not Modified

400 Bad Request

401 Unauthorized

403 Forbidden

404 Not Found

500 Internal Server Error

Status Code-Bytes GraphicStatus Code-Bytes Graphic

500-Internal Server Error Profile500-Internal Server Error Profile

Number of errorsNumber of errors

Average Time Taken By Different Average Time Taken By Different ErrorsErrors

ConclusionsConclusions

• By Nelson Model, the site software reliability is R = 0.966, or that 96.6% of access to website is successful.

• This model also shows that MTBF=29.6 hits or the site averages one error for every 29.6 hits.

• From the number of errors chart, we can see that Server errors are very few among the other errors which shows what the reliability of the DNN server is.

Conclusions

Our model Previous Model[1]

23 days data 26 days data

96.6 success 96.2 success

29.6 hits/error 26.6 hits/error

148,579 bytes per error

273,927 bytes per error

[1] J.Tian, S.Rudraraju, Z.Li, “Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs”,2004.

[2] T.Huynh, J.Miller, “Another viewpoint on 'Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs'”,2008.

[3] G. Albeanu, A. Averian, I. Duda, “Web Software Reliability Engineering”,2009.