1

ReStore Workshop: building more sustainable web resources

ESRC Research Methods Festival

5 July 2012

2

Introduction and objectives

• ReStore project• ReStore team members

– Jane Seale: Introduction– David Martin: The ReStore model– Arshad Khan: Building sustainable resources– Kaisa Puustinen and Katy Sindall

Objectives of the workshop

• Explain how ReStore works from the perspective of a potential author of an online resource

• Share lessons learned from ReStore’s experience of actual ESRC-funded research methods resource web sites

• Focus on approaches and specific actions which increase the sustainability of online resources - whether they are going into ReStore or not

• Opportunity to speak individually with you about your own resources

3

Format of the workshop

• Two-way interaction• Questions and discussion encouraged!• Time set aside for individual conversations at

end of morning• ReStore guidance materials to take away

4

The ReStore preservation and maintenance model

• ESRC funds a research methods project which creates an online resource (e.g. NCRM/RDI)

• Online part completed near to end of project funding• Often of great practical value, but immediately begin

to decay, lack of maintenance/visibility• Review and enhance the site, standardize as far as

possible but maintain ‘look and feel’ • Take on and maintain within ReStore repository

6

Why (specifically) ReStore?

• Repository for online resources• “Restoring”/enhancing quality and utility• Promoting accessibility• Sustainable service identity• Being implemented as an NCRM activity• NOT the only solution: if better options exist on a

project-by-project basis, that’s fine!

7

Not aims of the project

• A static web archive• A continuation funding model for completed

projects• A research methods advice service• A document repository• A virtual learning environment (*although

discussion later of Moodle/VLEs)

8

Learning from experience...

• Repository for completed online resources

• “Restoring”/enhancing quality and utility

• Promoting accessibility

• Development of guidance for existing ESRC projects

• Advice and practical assistance

9

10

ReS

toratio

n p

rocess

How does it work?

Review process

• Parallel technical, academic and author reviews• (i) Technical (ReStore team): site architecture,

scripting, portability, broken links, media types, potential IPR issues…

• (ii) Academic (external reviewers): academic content, rigour, referencing, dated material…

• (iii) Author: reflective review, cross-cutting technical and academic, esp. re. IPR

17

Technical review

• Architecture (stand-alone, CMS, VLE?)• Scripting (if any)• Scope (embedded within another site? Including

ephemera?) • Accessibility/standards/stylesheets• Metadata• Broken link checks

18

Academic review

• Academic ‘user’ review• Overall quality• Up to date?• Easily navigable?• Target audience?• References• (Similar to journal review)

19

Author review

• Author knows the strengths and weaknesses best

• Things we should know?• Often aware of incomplete

pages, uncertain about IPR issues, wants to fix things...

• (Akin to due diligence) 20

Consideration of reviews

• Decisions taken by ReStore team in editorial capacity, referred to advisory group re. matters of principle, e.g. precedents, difficult issues

• Review content summarised for authors• Team assessment of work required• Decision sets in train approved package of work• If not suitable, alternative strategies considered

– Static archiving, maintenance elsewhere, etc.

21

Commonly challenging areas

• Content and coverage • Metadata and access• Technology/software

considerations• Content management

systems

• Completeness, timeliness

• Usage monitoring• Team membership• IPR and ownership

considerations• Take-down policy

22

Design Implementation

• Content and coverage • Metadata and access• Technology/software

considerations• Content management

systems

• Completeness, timeliness

• Usage monitoring• Team membership• IPR and ownership

considerations• Take-down policy

23

Building more sustainable web resources: Part 1 - design

• Content and coverage• Metadata and access• Technology/software considerations• Content management systems

25

Content and coverage

• Is there an explicit learning design?– e.g. Geo-Refer learning objects and adaptive tutorials

• Is the material structured around a series of events?– Is this the best online presentation? What to keep?

Should it go into a document repository?

• Keeping ephemeral content easily separable– e.g. news, forthcoming events, etc.

26

Metadata and access

• Metadata “data about data” have a big impact on what search engines find– Best metadata created at source by the original

authors – very hard to do it well later!

• Access: is there really a good reason for restricting access?– NB significant overhead of maintaining user

registrations and login, data protection issues and restricts visibility of material

29

Metadata example

• Several medata standards available• Commonly used standard: Dublin Core• http://www.ukoln.ac.uk/metadata/dcdot/ • The above retrieves a web page and

automatically generate Dublin core metadata • title, creator, keyword, publisher, name, subject,

date one of the major DC elements.

30

EXAMPLE

Technology/software considerations

• Software used in creating the site• Tools for interactivity – e.g. web forms• Online security issues• Software assumed to be available to the user• Use of software examples, screenshots, code:

permissions required

32

Content management systems

• Increasingly the option of choice of universities:• Content Management Systems (CMS) for

building corporate websites – e.g. Teamsite• Virtual Learning Environments (VLE) for course

delivery – e.g. Blackboard• Externally hosted environments including

discussion Groups/Wikis – e.g. Google Groups• Proprietary and Open Source varieties of each...

33

Strengths Weaknesses

• Easy to set up new site• Little technical knowledge

required• User focuses on content• Someone else maintains

environment• Lack of control (good!)

• Limited design options• Reduced visibility of page

content to search engines, web archives

• Problems of access control (VLE, groups)

• Hard to export/move/ reformat content

• Lack of control (bad!)

38

http://code.google.com/apis/opensocial/

Extracting content from CMS

• Time consuming and prone to errors• Hard to reproduce “look and feel” of original• Very hard to reproduce proprietary functionality• Hard to identify and export all relevant files,

content, templates• Major programming task to move content• If must use, try to use Open Source (e.g.

Moodle, Drupal): content and system together42

Building more sustainable web resources: Part 2 - implementation

• Completeness, timeliness• Usage monitoring• Team membership• IPR and ownership considerations• Take-down policy

45

Completeness, timeliness

• Importance of finishing the project with a ‘complete’ resource and one that is up to date– References to literature, software, policy…

• Routinely checking for and fixing broken links increases utility and reduces user frustration

• Consider exit strategy: how will material be maintained after project end?

46

ESRC emphasis on impact...

• Yet most projects not monitoring usage!• Could start with seminar/workshop participants• Consider an online user survey, invite feedback• *May* get information from university server logs• More useful information from e.g. Google

Analytics – set up to monitor site/pages• Breakdown of web traffic (limited information

about users)48

ReStore user survey 2011

• Online survey (n=273)• 7.5% users from outside academic sector• 59% teachers, 25% researchers/research

students• 33% UK• 10% already knew about ReStore• 68% rating the site “very useful”• Textual feedback

49

Team membership

• Can be complex: multi-institution collaborations and external contributors, esp. once people move (and most do!)

• Is ownership of the material clearly articulated?• Has everyone following the same style?• Where did all the content come from?• Does PI/host institution have documented

permission for everything to be used/reused?52

Team issues...

• Ideal scenario: Original researcher still available, able to do own fixes in response to reviews: team members know own material best

• More commonly: Original researcher has left/ One or more investigators have moved institution (cannot access site!)/ External consultant built the website (requires payment to do any further work)/ Local IT staff not familiar with the site

53

Typical IPR scenarios

• All authors in same institution, ownership clear• Permissions required for images, papers,

screenshots, audio, video...• Issue of multiple academic contributors:

institutional affiliations during resource development?

• Very common: presentations from speakers at events – collect permissions at the time!

54

ReStore project individual contributor permission form

This is to confirm that I am the author of the __________________________________resource, which was originally created for, and has been hosted as part of the _____________________________________project website at the University of __________________.

I hereby give my consent for this resource to remain available as part of the project website, which is to be transferred into the ESRC-funded ReStore web repository (http://www.restore.ac.uk), currently hosted at the University of Southampton (although this may change to another academic institution at some future date).

I also confirm that I am not aware of any third party implications that prevent me from giving this consent

Signed ____________________________ Name ____________________________ Date _______________________

IPR Principles

• Maintain IPR register• Permissions for all content types• Record of contributor permissions • Contractual arrangements between institutions• Mostly common sense!• Examples on ReStore guidance pages

57

ReStore deposition licence

• License from the host institution to ReStore (University of Southamtpon) to host a resource

• Undertakes to have undertaken due diligence checks for obtaining contributors’ permissions and citing sources

• Permits editorial revisions, updates, metadata, reformatting, etc.

• Permits display of licensor’s logo, trademark etc. • Original authors and project clearly identified

58

Take-Down Policy

• Mitigates risks from IPR infringement• Makes process clear• Who to contact• What action will be taken• Thinking it through ensures consideration of how

such an issue would be handled• We have had to use it!

59

Policy and guidance

• At present, no ESRC “web resources policy”• Increasing emphasis on institutional repositories,

but will long development path ahead• Keep things open: consider open educational

repository• No reason not to adopt best practice now• Support and guidance available• See also: ESRC Research Data Policy

61

Discussion of practical challenges

64

The project life-cycle

• Conception• Proposal• Project• Resource creation• Post-award

– The earlier these issues are addressed, the easier they are to resolve

65

Issues for award holders to consider

• How ready are you for project end!?• Technical design issues: are online materials

accessible, transferrable, documented?• Are all organizational issues resolved and

recorded, licensing sorted? Is it open?• Sustainability – Where will it go next? What will

the legacy be? Who will manage it after funding? • Do you have the right tools and data for impact?

66

Next steps

• Contact us• Follow up specific questions• Use the ReStore guidance pages• www.restore.ac.uk/guidance• Presentations from Research Methods Festival

workshop• http://www.ncrm.ac.uk/RMF2012/

67