A centre of expertise in digital information management
www.ukoln.ac.uk
1
Preserving Project Web Sites: The Lessons Learnt
Brian Kelly
UKOLN
University of Bath
BATH
UK
Email: [email protected]
UKOLN is supported by:
http://www.ukoln.ac.uk/web-focus/events/workshops/dcc-2006-01/http://www.ukoln.ac.uk/web-focus/events/workshops/dcc-2006-01/
This talk provides pragmatic advice on the short-medium term preservation of project Web sites and the application of this advice to other areas
This talk provides pragmatic advice on the short-medium term preservation of project Web sites and the application of this advice to other areas
A centre of expertise in digital information management
www.ukoln.ac.uk
2
Contents• Why Is Web Site Preservation An Issue?• The Nightmare Scenario• Administrative Issues• Technical Challenges • What Is My Web Site?• What Is My Preferred Future For My Web
Site?• Mothballing Procedures• Lessons For Future Work• Questions
A centre of expertise in digital information management
www.ukoln.ac.uk
3
Why Is Web Site Preservation An Issue?Digital Resources Don't Rot
• Digital resources (images, video, software, Web sites, …) don't degrade due to environmental factors. This is a key difference with physical resources.
• Web sites are made from various digital resources: HTML pages, GIF, JPEG, etc. image files, PDF resources, software (CGI scripts, JavaScript, etc.)
• These won't degrade so why is Web site preservation an issue?
• Isn't the fact that old Web sites won't disappear and may be embarrassing more of a challenge?
A centre of expertise in digital information management
www.ukoln.ac.uk
4
The Nightmare Scenario
To be avoided:• The funding finishes• Project staff leave, partnership dissolves• Hosting agency upgrades operating system,
resulting in scripts to access resources from backend database are broken
• User finds page with invitation to project launch and travels to meeting. Unfortunately the event took place in 2002.
• Invoice for domain name is not paid, as administrator has left.
• Web site domain taken over by porn company• Prime Minister picks up pen containing project
URL and visits pornographic Web site
A centre of expertise in digital information management
www.ukoln.ac.uk
5
It Has Happened!Webtechs.com
• Software company which hosted early HTML validation service
• In 1998/99 confusion over payment of domain name
• March 1999 company receives many messages saying validation service is now a porn site
• Over 30,000 links to Web site!
• Sept 1999 porn company agrees to sell domain name back to Webtech
http://www.webtechs.com/http://www.webtechs.com/
A centre of expertise in digital information management
www.ukoln.ac.uk
6
The Embarrassment Still ExistsThe hijacked Web site can still be accessed using the Internet Archive's Wayback Machine.
Note that the archived Web site contains JavaScript (and Active X controls?) which could delete data on the viewer's PC
See <http://www.exploit-lib.org/issue1/webtechs/>See <http://www.exploit-lib.org/issue1/webtechs/>
Warning!
Who is responsible if this software deletes files?
A centre of expertise in digital information management
www.ukoln.ac.uk
7
A Possible Scenario
A potential scenario:• Project Web site developed• Organisation has limited networking expertise• Domain name lapsed due to lack of knowledge of
terminology ("What's a DNS? Is this invoice legit?")
• Once virtual domain name lapsed, accesses go to service developer's Web site
• Developer's Web site has links to Web sites they've built (including some of a dubious nature)
• Once address expires in DNS caches links go to a porn company
• Funder's gateway points to a porn site!
Let's ensure that this doesn't happenLet's ensure that this doesn't happen
A centre of expertise in digital information management
www.ukoln.ac.uk
8
A Web Site Isn't Just For Christmas, It's For life!The lessons:
• You need to be aware that Web sites developed using short-term project funding need to be kept for a long period after funding finishes
• Porn domain name pirates are looking for Web sites whose domain name has expired
• Web sites which are well-linked and easily found using Google are particularly attractive to porn pirates
• You will want to avoid this happening to your Web site
• You will want to ensure your Web site doesn't link to sites which transform to dodgy sites
A centre of expertise in digital information management
www.ukoln.ac.uk
9
Other Administrative IssuesDigital Signatures
• You buy a digital signature which identifies your Web site as belonging to a legitimate organisation
• The digital signature used for (a) the encryption of credit card details and (b) use of an Intranet
• You fail to renew the signature / renewal not accepted as the consortium is not a legal entity
• Users see "Non-valid signature" message
http://www.bathimpact.com/io_article.php?section=On%20Campus&ref=48
http://www.bathimpact.com/io_article.php?section=On%20Campus&ref=48
A centre of expertise in digital information management
www.ukoln.ac.uk
10
What Is My Web Site?
What do we mean by my Web site?
What purposes could be provided by my Web site?• The public Web site which users see• Several Web services used by users (e.g.
www.foo.org.uk, search.foo.org.uk, …)• The Web site containing a public area and a
private area for use by consortia members• A public Web site and a private one• Several public Web sites, one for each member of
the consortia
See <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-15/>
See <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-15/>
A centre of expertise in digital information management
www.ukoln.ac.uk
11
The Preferred Future For My Web SiteAfter the project funding finishes:
• The project money has helped pump-prime an activity which is core to my organisation's mission. The project Web site will be developed through my organisation's existing funding streams.
• We'd like to build on the work. We're looking for new funding streams.
• We've decided we don't want to engage in the e-world. We'd like someone to take the Web site off our hands (we don't want it to become a porn site!)
• We haven't given any though to this. Anyway we're all left the project.
A centre of expertise in digital information management
www.ukoln.ac.uk
12
Technical Issues
Standards And Formats• Has the Web site been designed using open
standards, which should help future-proofing?• Have proprietary formats been used (for which
backwards compatibility may not be considered)?
Architecture & Implementation• Has the technical architecture of the Web site
been documented?• Can I continue to use technical systems after
funding has finished?
A centre of expertise in digital information management
www.ukoln.ac.uk
13
Mothballing Your Web Site (1)
Before funding finishes you should take steps for the mothballing of your Web site:
• Run a link check across the Web site. Fix broken internal links and as many external links as is reasonable. Document the link report.
• Run HTML (and CSS) validation checks across the Web site. Fix as many invalid pages as is reasonable. Document the findings.
• Run an accessibility check across the Web site. Fix as many inaccessible pages as is reasonable. Document the findings.
This should not be an onerous task if you have following best practice guidelines. Note that errors found later occurred after your funding finished.
This should not be an onerous task if you have following best practice guidelines. Note that errors found later occurred after your funding finished.
A centre of expertise in digital information management
www.ukoln.ac.uk
14
Mothballing Your Web Site (2)
You should also address technical areas:• Remove any backend scripts which are no longer
needed (e.g. online booking forms for old events).• Remember that scripts, etc. are liable to go wrong.
Ensure that applications are configured to break gracefully and provide meaningful errors: The config.ssi is missing. This should be
reported to the systems administrator (email [email protected] or ring +44 020 123 123. Please provide the URL of the broken page and the project name)
Apache error 6963You'll have to ensure that you have procedures to maintain this information
A centre of expertise in digital information management
www.ukoln.ac.uk
15
This Web si
te is no lo
nger maintained.
This Web si
te is no lo
nger maintained.
See home page for d
etails
See home page for d
etails
Mothballing Your Web Site (3)
You should also address the content of your Web site:• Clarify the status of the Web site on the home
page.• Ensure the tense of the content reflects the
position i.e. don't say "This project will …"• Ensure that contact details will remain valid i.e.
provide generic email addresses not an individuals
• Remember that many users will arrive deep in your Web site (e.g. using Google). If necessary use CSS to flag all pages with a watermark
See <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-04/>
See <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-04/>
A centre of expertise in digital information management
www.ukoln.ac.uk
16
Changing Web Site Address
What can happen:• Project finishes and project URL changes• Links to Web site break• Content appears to disappear
What should you do?• Plan from the start of the project!• Clarify purpose(s) of Web site• Remember Tim Berners-Lee's advice:
"Cool URIs don't change"
See "Changing a Project's Web Site Address" at<http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-
32/>
See "Changing a Project's Web Site Address" at<http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-
32/>
A centre of expertise in digital information management
www.ukoln.ac.uk
17
Mothballing Toolkit
UKOLN and AHDS have developed a QA framework to support JISC's digital library programmes
The JISC-funded QA Focus work included a simple lightweight automated self-assessment toolkit
http://www.ukoln.ac.uk/qa-focus/toolkit/mothballing-01/
http://www.ukoln.ac.uk/qa-focus/toolkit/mothballing-01/
A centre of expertise in digital information management
www.ukoln.ac.uk
18
Testing Repurposing Of Your Web Site You may find that:
• Your Web site is repurposed by third parties• You wish to move your Web site to another location
In order to check that repurposing can happen without errors you should think about testing the process:
• If you have a PDA use Avantgo.com (or similar) tool to access Web site on another device
• Use a Web site mirroring tool (e.g. HTTrack) to copy your Web site to your desktop PC
Such tools can:• View your Web site will look on other devices• Spot potential problems for mirroring your Web site
See <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-05/>
A centre of expertise in digital information management
www.ukoln.ac.uk
19
The Copyright Problem
Someone else will archive your Web site?
During 2005 national archiving initiative requested deposit of project Web sites
Form required statement regarding copyright ownership
But:• Who owns the copyright? (UKOLN, Univ
of Bath, UKOLN staff, other orgs, other individuals, ….)?
• Can we sign the form?
A centre of expertise in digital information management
www.ukoln.ac.uk
20
Lessons For The Future
How easy is it for you to implement mothballing techniques?
• You may find that deploying a watermark on every page of your Web site is time-consuming to implement
• Any difficulties encountered with your project should be noted and lessons learnt should applied to future development work
• Think about preservation from the original planning stage for a Web site
A centre of expertise in digital information management
www.ukoln.ac.uk
21
Case Study - Exploit Interactive (1)Exploit Interactive:
• EU-funded ejournal available at <http://www.exploit-lib.org/>
• Funded from Jan 1999 – Dec 2000• Web site is still hosted locally
Issues:• Should we continue hosting domain after 3
years?• What is the cost of this (domain name
registration, disk storage, system maintenance)?
A centre of expertise in digital information management
www.ukoln.ac.uk
22
Case Study - Exploit Interactive (2)Findings:
• Disk storage is 4Gb (large proportion is log files)• A 30 Gb disk drive cost ~ £40 (now cheaper)• Annual link check to be carried out. • Estimated that it would take about 30 minutes /
year to run a link check and document findings.• Policy for ongoing hosting of Web site agreed• See <http://www.ukoln.ac.uk/qa-focus/documents/case-studies/case-study-17/>
• See annual surveys (Exploit & Cultivate)• Note that e-journals still being accessed
A centre of expertise in digital information management
www.ukoln.ac.uk
23
Short-Medium Term Access Policy
We will:• We will seek to ensure the Web site continues for at
least 10 years after the end of funding.• We will seek to ensure that the Web site continues to
function.• We will not fix broken links to external resources.• We will not fixing non-compliant HTML resources.
We will use the following procedures:• We will have internal administrative procedures to
ensure that the domain name bill is paid.• We will record disk space usage and provide an
estimate of the cost of providing disk space • We will run a link checker annually and record the nos.
of internal broken links. We will keep an audit trail to see if internal links start breaking.
• Any changes to the policy … need to be agreed by an appropriate management group.
A centre of expertise in digital information management
www.ukoln.ac.uk
24
Conclusions
To conclude:• Web sites can disappear• They may reappear as porn sites!• Organisations should ensure they have
procedures to ensure this does not happen• You should developed a medium term
Web site preservation strategy • You should test mirroring of your Web site• You should seek to address such issues at
the planning stage of your Web site
A centre of expertise in digital information management
www.ukoln.ac.uk
25
Questions?
Any questions?
Top Related