Issues in Human Rights Web Archiving
description
Transcript of Issues in Human Rights Web Archiving
Human Rights Archives and Documentation, CHRDR Conference 4-6 October 2007
Issues in Human Issues in Human RightsRights
Web ArchivingWeb ArchivingRobert WolvenRobert Wolven
Columbia University LibrariesColumbia University Libraries
Libraries have a mission to build, Libraries have a mission to build, organize, and preserve coherent organize, and preserve coherent collections for researchcollections for research
There’s a great deal of human rights-There’s a great deal of human rights-related content on the webrelated content on the web
Much of it is not currently collected Much of it is not currently collected by librariesby libraries
Something should be done about thatSomething should be done about that
A great deal of content exists only A great deal of content exists only onlineonline
There’s a high risk that some will There’s a high risk that some will disappeardisappear
Libraries and archives are custodians Libraries and archives are custodians of our cultural heritageof our cultural heritage
Libraries and archives should lead in Libraries and archives should lead in preserving “at risk” contentpreserving “at risk” content
Web Archiving as Web Archiving as PreservationPreservation
Small footprint in organizationSmall footprint in organization
The Hoover effectThe Hoover effect
Haphazard library collectionsHaphazard library collections
Ineffective accessIneffective access
A Lot of ContentA Lot of Content
Much of it is not … Much of it is not … collectedcollected
Refugees InternationalRefugees International 40 documents on web site40 documents on web site
0 in Columbia collections0 in Columbia collections
10 listed in OCLC10 listed in OCLC 1 held by more than 2 libraries1 held by more than 2 libraries No library holds more than 3No library holds more than 3
Web Archiving IssuesWeb Archiving Issues
Ways and MeansWays and Means Selection policiesSelection policies Permissions – and ObligationsPermissions – and Obligations Organization and IntegrationOrganization and Integration Presentation and UsesPresentation and Uses Sharing the Costs and BenefitsSharing the Costs and Benefits Organizational TransformationOrganizational Transformation
Center for Research Libraries’ Center for Research Libraries’
Political Communications Web Archive Political Communications Web Archive ProjectProject
Project website: Project website: http://www.crl.edu/content/PolitWeb.hthttp://www.crl.edu/content/PolitWeb.htmm
Final report: Final report: http://www.crl.edu/PDF/PCWAFinalReport.pdfhttp://www.crl.edu/PDF/PCWAFinalReport.pdf
Web Archiving ToolsWeb Archiving Tools
Archive-It (Internet Archive) Archive-It (Internet Archive) http://www.archive-it.orghttp://www.archive-it.org
PANDAS (National Library of PANDAS (National Library of Australia)Australia) http://http://pandora.nla.gov.au/pandas.htmlpandora.nla.gov.au/pandas.html
OCLC Digital ArchiveOCLC Digital Archive http://www.oclc.org/digitalarchive/defaulhttp://www.oclc.org/digitalarchive/defaul
t.htmt.htm
““I only want to download I only want to download text/html and nothing else. text/html and nothing else.
Can I do it?”Can I do it?” You can … add a filter that excludes You can … add a filter that excludes
all filters that end in other than 'html|all filters that end in other than 'html|htm', etc., or, if you want to instead htm', etc., or, if you want to instead look at document mimetypes, you can look at document mimetypes, you can Add a ContentTypeRegExpFilter filter Add a ContentTypeRegExpFilter filter as a midfetch filter to the http as a midfetch filter to the http fetcher.fetcher.
[From Heritrix FAQ [From Heritrix FAQ http://crawler.archive.org/faq.html#user-heritrix]http://crawler.archive.org/faq.html#user-heritrix]
Policy Policy Technology Technology
Full web site or selected content?Full web site or selected content?
Preserve relationships, “look and Preserve relationships, “look and feel”?feel”?
All file types?All file types?
How often?How often?
From the European Union Agency for Fundamental Rights website
“CorrigendumChanges have been made to the report on
"Muslims in the European Union – Discrimination and Islamophobia" after it was printed.
Following pages are replacing the Annex page in the EN and FR version of the report. PDF”
Selection by Type of Selection by Type of AgencyAgency
GovernmentalGovernmental
InternationalInternational
AcademicAcademic
EducationalEducational
Selection by FocusSelection by Focus
Global, regional, localGlobal, regional, local
Ethnicity, religion, gender, ageEthnicity, religion, gender, age
Legal, medical, economicLegal, medical, economic
Crisis-drivenCrisis-driven
Selection by ContentSelection by Content
Fixed documents:Fixed documents: Case studiesCase studies Position papersPosition papers Topical reportsTopical reports Press releasesPress releases Bulletins, newslettersBulletins, newsletters Activity reportsActivity reports
Selecting by ContentSelecting by Content
Non-textual (image, sound, video)Non-textual (image, sound, video)
Ephemeral, dynamic contentEphemeral, dynamic content
Redundant (?) content:Redundant (?) content: Languages, formatsLanguages, formats Republished or unique?Republished or unique?
Rights and ObligationsRights and Obligations
Permissions: ask or assumePermissions: ask or assume Rights:Rights:
Dark archiveDark archive Closed archiveClosed archive Conditional exposureConditional exposure
Obligations:Obligations: Parallel (mirror) accessParallel (mirror) access Free, reliable accessFree, reliable access Perpetual access Perpetual access
Organization and Organization and IntegrationIntegration
… … or, now that we have it, how do we or, now that we have it, how do we know what we’ve got? know what we’ve got?
How do other agencies know what’s How do other agencies know what’s been done?been done?
How do researchers find it?How do researchers find it?
“From 1 March the European Monitoring Centre on Racism and Xenophobia (EUMC) became
the EU Agency for Fundamental Rights (FRA). The content on the website is being gradually transformed to reflect the scope, activities and
products of the new Agency.”
Integrating AccessIntegrating Access
Through Authority ControlThrough Authority Control
Through controlled vocabularyThrough controlled vocabulary
Through seriesThrough series
Integrating CollectionsIntegrating Collections
With print – in the catalogWith print – in the catalog
With archives – in finding aidsWith archives – in finding aids
With digital collections – in …With digital collections – in …
UseUse
Internal organization and navigationInternal organization and navigation
IndexingIndexing
Analytical toolsAnalytical tools
Citation: pedigree and persistent Citation: pedigree and persistent linkslinks
Sharing Costs and Sharing Costs and BenefitsBenefits
Centralized CollaborationCentralized Collaboration
Distributed CollaborationDistributed Collaboration
Disclosure (at what level of detail)Disclosure (at what level of detail)
Exposure (to the web; OAI-PMH)Exposure (to the web; OAI-PMH)
Transformative ActionTransformative Action
Concept of “collecting”Concept of “collecting”
Modes of selectionModes of selection
Bridging communities of practiceBridging communities of practice
““Where do you Where do you stop?”stop?”