A centre of expertise in digital information management QA For Web Sites Brian Kelly UKOLN...

A centre of expertise in digital information management www.ukoln.ac.uk

QA For Web Sites

Brian KellyUKOLNUniversity of BathBath

[email protected]

URLhttp://www.ukoln.ac.uk/

UKOLN is supported by:

Contents • What Is Quality • Quality And Web

Sites• QA Case Study• Approaches To

Testing• Benchmarking• Conclusions

Contents • What Is Quality • Quality And Web

Sites• QA Case Study• Approaches To

Testing• Benchmarking• Conclusions

A centre of expertise in digital information management www.ukoln.ac.uk 2

Background To Web & QA Focus

Web Focus:• A JISC-funded post to advise the UK’s HE and

FE community• Provided by Brian Kelly

QA Focus:• An advisory service for JISC’s 5/99 projects• Provides advice and support for projects in order

to ensure that projects comply with standards and best practices

• Provided by UKOLN (Marieke Guy & Brian Kelly) and ILRT

This talk is informed by the work of Web and QA Focus


Quality Control and Quality Assurance

Quality Control (QC):• Use of regular testing procedures against your

definitions of quality and more specifically the refinement of these procedures

• Relates to outputs

Quality Assurance (QA):• examines the processes that shape your Web

site in the first place and so encompasses Quality Control

• Relates to inputs• Should be involved in development stage of a

Web site and throughout its life cycle


What Can Fail on Your Site?

• Broken links, broken emails• Server load – too many hits on the site• Client side performance –down load time• Security isn’t working• Content is out of date• Browser incompatibility,HTML doesn’t validate• Interface – navigation, link colour• Graphics missing or too large• Scripts don’t work - forms, databases• Isn’t accessible to those with disabilities• Browser dependencies• …


What Can You Test?• Functional testing • Compatibility testing• Load/performance testing• Stress testing • Usability testing• Security testing• Integration of unit testing• Link testing• HTML Validation• Reliability testing• Regression testing• …


What Procedures?

• Good documentation• Requirements• Specifications• Mission statements or statement of direction

• Define your audience • Testing suite and tools• Usability testing• Use the right authoring tools etc.• Track problems


Issues

Things to be aware of:• Testing goals should relate to kind of site you have• Testing time is limited use automated tools • Automated tools can be inadequate use manual

tests• Documentation is critical to ensure practices are

repeatable• Motivating/persuading people to implement a QA

culture can be difficult• Implement your QA procedures over time to avoid

stifling productivity• QA planning at the start of development means

less time fixing things at the end


Case Study: IWMW Web Sitehttp://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/

Cas

e S

tud

y


Aims Of Web Site

The aims of the Web site were:• To provide information about the workshop for

participants, speakers, etc.• To provide an online booking facility• To act as a demonstrator of standards, best

practices and innovative solutions

Subject to the following constraints:• No Content Management System• Limited time• Limited software development / design expertise

Cas

e S

tud

y


ApproachThe approach taken was to use:

• The design and structure used for previous two workshops

• The HTML-Kit authoring tool to edit HTML files• The Xenu link checker

Innovations this year include:• Use of HTML and CSS validation icons• Provide an XHTML Web site• Make greater use of CSS• Provide a dynamically-generated RSS news feed• Provide live usage statistics• Access to Web site on PDAs using AvantGo

Is this approach typical of yours – incremental development of an existing Web site, and introducing new features and more up-to-date standards?

Is this approach typical of yours – incremental development of an existing Web site, and introducing new features and more up-to-date standards?

Cas

e S

tud

y


ToolsHTML-kit is my preferred editor (and recent version is even better, providing batch validation)

The XHTML and CSS icons act as live linksThe SiteMeter icon gives realtime info on usage

The XHTML and CSS icons act as live linksThe SiteMeter icon gives realtime info on usage


Problems

Some problems:• When updating pages, links, HTML and spelling

were not checking systematically• When using the HTML and CSS checker the

browser could check an old, cached version• At one stage the SiteMeter usage statistics

service was not available

Cas

e S

tud

y

It can be a useful (and therapeutic!) process to talk about problems with your Web site.

It can be a useful (and therapeutic!) process to talk about problems with your Web site.


Validating In Batch

Checking (e.g. HTML and CSS validation, links, accessibility, etc.) initial files is time-consuming and, for large or continually updated Web sites is not a scalable solutionValidation in batch:

• Familiar for link-checking• Limited set of tools available for HTML validation

(e.g. HTML Validator Pro> • Batch HTML validation validates files generated

from PowerPoint, over which I have no control• Bobby batch accessibility check is now a licensed

application – so used old copy

Cas

e S

tud

y

Can now use HTML-Kit for validating the Web site.Can now use HTML-Kit for validating the Web site.


Netscape (sigh)

User feedback revealed that:• £ does not display in Netscape :-(• The Suggestions text box and the Submit button

(!) were not displayed in versions of Netscape, due to failure to support XHTML correctly

Suggestions

Submit

Cas

e S

tud

y


RSS News FeedThe RSS News Feed:

• Provides a mechanism for syndication of content• News feed can be viewed on other Web sites or

in other applicationsThe News Feed was created:

• By transforming (lightly structured) HTML from the news page using a remote service

• A static version was also created in case of problems with the live transformation

Problems:• The remote service was not always available • The static version was not always in sync with the

live version

Cas

e S

tud

y

Subsequently the dynamic transformation service became unavailable

Subsequently the dynamic transformation service became unavailable


Online Bookings

The online booking forms:• Were used for booking for the workshop and choosing the

parallel sessions

The backend processing:• Is not yet fully automated to store the information in a

database• Confirmation page did not echo the user’s data• When users followed link to check session, input data may

have been lost• Crashed on surnames with spaces and accents• No validation was performed

Despite messages on acknowledgements page, some users still expected a separate email confirmation

Cas

e S

tud

y


What Should Be Done

Some simple things can be done for next time:• Use of SSIs or a CMS (or equivalent ) to manage

resource fragments• Improve the online booking system to provide:

• Validation (which will save time in administration)• Display of user input• Separate confirmation of entry into database

But:• I recognise that minor typos, etc. will happen and,

as long as they are not critical, can be tolerated• The goal is quality – not perfection

Cas

e S

tud

y


What About The Processes?

What QA processes were used?• Automated tools• Peer review from members of the organising

committee• Alternating between Opera, IE and Mozilla when

working on Web site• Viewing the Web site on my Palm to:

• Allow me to work on the bus, train, etc.• See how the Web site works on a low-spec device

See <http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/qa/>

See <http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/qa/>

Cas

e S

tud

y


What Else Should I Do?

Additional processes I should employ:• Reviewing by users• Peer-reviewing (others in similar situation)• Analysis of logs (esp. failure logs)• Usability testing• Multiple browser testing• Test data with unusual entries for online forms• Post-workshop review of Web site• Systematic documentation of successes and failures• Make case for extra resources, software, etc. to

implement improvements• Documenting the architecture of the Web site• Documenting the limitations of the Web site• Preparing these slides!• …

Cas

e S

tud

y


Automated TestingAutomated Testing:

• Should be scalable• Only suitable approach for large Web sites• Can push (email) information about problems• Many free automated testing tools available

but:• Will not spot all usability / accessibility problems• Can provide too much information• Comprehensive and configurable testing tools

can be expensive


Manual TestingManual Testing:

• Can spot usability / accessibility problems• Can make use of one’s community / user base /

peers• Can use of key pages which will identify

problems which occur elsewhere • Particularly useful for testing new Web sites• “5 users can spot most of your problems”

but:• Not suitable for testing large numbers of pages• Quality of feedback may be variable• Often not useful for testing established Web

sites


Approaches To Automated Testing

Automated testing can make use of:• Desktop tools• Web-based tools

UKOLN has made use of Web-based testing tools:• Monitor accessibility, HTML, CSS, compliance,

page size, links quality, nos. of links, etc.• Benchmarking approach provides comparison

with one’s peers• See, for example,

<http://www.ukoln.ac.uk/web-focus/events/conferences/ucisa-tlig-2002/benchmarking/>


Interfaces to Testing Tools (1)

Bookmarklets• Embed a testing service into your browser• Available for IE, Netscape, Opera, …browsers• Article at <http://www.ariadne.ac.uk/issue19/web-focus/>

• See <http://www.bookmarklets.ac.uk/> or Google search for “bookmarklets”, “accessibility bookmarklets”, etc.


Bookmarklets Example

Bookmarklets can be accessed:

• From the browser’s normal Bookmarks / Favourites Menu

• From a side bar• By right-clicking

the mouse button


Interfaces to Testing Tools (2)

URL Interface• UKOLN’s Web site has an interface to various

tools provided at the server• Easier to maintain• See <http://www.ukoln.ac.uk/,tools>

for overview

<http://www.ukoln.ac.uk/,validate><http://www.ukoln.ac.uk/,validate>


Benchmarking (1)

Benchmarking:• We have discussed various tools for checking

one’s own site• But how do we compare with our peers?• Can we learn from others best practices?• Can we avoid making mistakes that others have

made?

Can the methodology to be described be used across your community – e.g.

• On a regional basis• On candidates for Best Public Library Web Site award • By government auditors for checking e-GIF compliance

Can the methodology to be described be used across your community – e.g.

• On a regional basis• On candidates for Best Public Library Web Site award • By government auditors for checking e-GIF compliance


Benchmarking (2)


QA Focus Web Site Benchmarking

QA Focus:• Carried out survey of 43 project Web sites• Findings freely available• Methodology can be applied to other sectors• Where best practices found, projects asked to

provide a case study• Where problems found, appropriate advice

provided


HTML Compliance

W3C’s HTML Validation service is used to record the HTML compliance of project home pages

http://www.ukoln.ac.uk/qa-focus/surveys/http://www.ukoln.ac.uk/qa-focus/surveys/


Accessibility

Bobby is used to record compliance with W3C WAI guidelines:

24 comply with A

5 comply with AA

21 are not compliant

Note that only compliance with guidelines which can be monitored by automated tools is recorded

Note that only compliance with guidelines which can be monitored by automated tools is recorded


Repurposing Resources

Are pages from project Web sites available in the Internet Archive?

Can the project Web site be accessed on a PDA?


404 Pages

404 error pages can provide an important navigational feature for Web sites

Tailored 404 Page 14 33%

Default 404 Page 28 67%

* Rating 3 21%

** Rating 3 21%

*** Rating 8 57%

**** Rating 0 0%

* Basic** Simple branding*** Additional functionality**** Full functional


Critique

Limitations of this approach include:• Project Web sites may be:

The project deliverable About the project For intra-project communications

• The project deliverable may be, say, middleware • Limitations of the analysis tools• Limited number of pages surveyed• …


Supporting Best Practices

QA Focus is supporting JISC 5/99 projects by providing advice on compliance with standards and best practices, based on the survey findings

http://www.ukoln.ac.uk/qa-focus/documents/

http://www.ukoln.ac.uk/qa-focus/documents/


Case Studies

QA Focus is commissioning case studies:

• Document approaches projects have taken in particular areas

• Describe the problem area, the approach taken, problems experienced and lessons learnt


Conclusions

To conclude:• Quality assurance is important• It will grow in importance: e.g. compliance with e-

GIF guidelines, accessibility legislation, etc.• Automated tools can help• Manual testing is needed to complement

automated approaches• Benchmarking can provide information on

approaches across communities

A centre of expertise in digital information management QA For Web Sites Brian Kelly UKOLN...

Documents

Transcript of A centre of expertise in digital information management QA For Web Sites Brian Kelly UKOLN...