Update on the Fedora ProjectWhere we’ve been and where we’re going

Fedora Users ConferenceRutgers UniversityMay 13-14 2005

Sandy PayetteCo-Director Fedora Project

Cornell University

Fedora Brief History

• Cornell Research (1997-present) – DARPA and NSF-funded research– First reference implementation developed– Interoperable Repositories (experiments with CNRI)– Policy Enforcement

• First Application (1999-2001)– University of Virginia digital library prototype – Technical implementation: adapted to web; RDBMS

storage– Scale/stress testing for 10,000,000 objects

• Open Source Software (2002-present)– Andrew W. Mellon Foundation grants– Technical implementation: XML and web services– Fedora 1.0 (May 2003)– Fedora 2.0 (Jan 2005)– Fedora 2.1 (coming soon!)

Fedora Development Team

Cornell University

• Sandy Payette (co-director)• Chris Wilper• Carl Lagoze• Eddie Shin

University of Virginia

• Thorny Staples (co-director)

• Ross Wayland• Ronda Grizzle• Bill Niebel• Bob Haschart• Tim Sigmon

“Fedora Inside” Known Use Cases

• Digital Library Collections• Institutional Repository • Educational Software

• Information Network Overlay • Digital Archives and Records Management• Digital Asset Management• File Cabinet / Document Management• Scholarly publishing

Fedora Repository 2.0

M anage AuthN AuthZ

Access Validation Resource Index

Storage Dissemination Fie ldSearch

Fedora Repository M odules

M anag e A c c e s s Se arc hR ISe arc h(R D F )

R E S T

C lie n tA pp

Ba tchProgram

O th e rS e rv ice

W e bBrows e r

R E S T S O A PS O A P R E S T S O A PR E S T

O A IP ro v ide r

R E S TWeb Services

Exposure

Fedora – what’s new(version 2.0)

• FOXML (Fedora Object XML)– Simple XML format directly expresses Fedora object model– Easily adapts to Fedora new and planned features– Easily translated to other well-known formats

• Enhanced Ingest/Export of objects– FOXML, METS (Fedora extension)– Extensible to accommodate new XML formats– Planned: METS 1.4, MPEG21 DIDL

Fedora 2.0 (continued)

• Object-to-object Relationships– Ontology of common relationships (RDF schema)– Relationships stored in special datastream (RELS-EXT)

• Resource Index (RI)– RDF-based index of repository (Kowari triple-store)– Graph-based index includes:

– Object properties and Dublin Core– Object Relationships– Object Disseminations

• RI Search (Search the repository as a graph)– Powerful querying of graph of inter-related objects– REST-based query interface (using RDQL or ITQL)– Results in different formats (triples, tuples, sparql)

info :fe do ra/im age :1 1

la stM odDate

hasMem ber

hasM em ber

hasR ep

hasR ep

info:fe dora/ imag e :1 1 /B LDG

in fo:fe dor a/ imag e :1 1 /bde f:2 /g e tR e latedLe tte r

hasRep

i n fo:fe dor a/c ol l e c tion :1 / bde f:1 /MEMB ERS

info :fe do ra/im age :1 2

in fo :fe do ra/c o lle c tio n :1

la stM odDatehasR ep

"2 0 0 5 - 0 1 - 1 0 :1 1 :0 2 "

"2 0 0 5 - 0 2 - 0 1 :1 2 :0 5 "

lastModD

a te

"2 0 0 5 - 0 1 - 0 1 :1 0 :0 0 "

dc :c rea

tor

"E lly C r am er "

dc:crea to r

"C h r is W ilp er "

in fo:fe dor a/ imag e :1 2 /B LDG

dc:creato r

"E d d ie S h in "

in fo:fe dor a/ imag e :1 2 /bde f:2 /g e tHIGH

hasR ep

Fedora Objects – RDF Graph view

CollectionObject

MemberObject

Fedora 2.0 (continued)

• New Utilities• Batch Modify Utility • Repository Administrator Reporting

• Performance Tuning (~1 million objects)– Ingest testing (800K objects; 40 millisec/object)– Concurrency testing (access requests)

• Communications and Outreach• New Fedora Web Site• Improved Documentation• Tutorials

Preview Fedora 2.1 (May 2005)

• Authentication plug-ins– HTTP Basic auth – Tomcat realms and login modules

• Plug-in #1 : Tomcat user/password file or database• Plug-in #2 : LDAP tie-in• Plug-in #3 : Radius Authentication

• Support for SSL

• Authorization module– XML-based policies using XACML– Repository-wide policies– Object-specific policies– Fine-grained policy enforcement

• API actions X subject attributes X object attributes

Authorization: Example Policies

• Repository Policy– Deny access to all API-M methods if not “administrator” – Deny access to the deleteDatastream method if not “author”

• Specific Object Policy– Deny access to object “uva:100” if user is not Thorny.

• Group Object Policy– Deny access to the “getFullArticle” dissemination of objects

whose content model is “journal-article” if not “faculty”– Deny access to the “secret” datastream if user is “guest”

Authorization: Example Policies

• Time-oriented Policy– Permit students access to “answers” datastream of

learning object cs:125 after May 15, 2005– Deny all access learning object cs:125 after June 15, 2005

• Backend Service Security Policy– Deny callback by external service represented by

Bmech:10

Preview Fedora 2.1 (May 2005)

• Policy Builder Client– Simple user interface to create access policies– Automatically generate XACML

• Handle Generation Plug-in (PIDs as Handles)

• Enhanced OAI Provider Service (prOAI)– Harvest multiple metadata formats– Harvest datastreams and disseminations– Support for incremental harvest by modified date– Support for OAI sets– Highly configurable via queries against Resource Index

• Directory Ingest Service (and client tool)– Facilitate ingest of hierarchical directories of files– Submit files as .zip or .jar (with a METS manifest)– Automatically asserts parent-child relationships in RELS-EXT– Stages content and ingests as FOXML objects into repository

Fedora Service Framework(beginning Fedora 2.1)

F e d o ra Re p o sito rySe rv ice

OAIProvide rSe rvice

Dire ctoryInge st

Se rvice

Futur e

Ser v ice

Serv ices

Apps

Other

Ser v ice

Administrator

Other

Ser v ice

Futur e

Ser v ice

PolicyBuilder

ZIP or JARinput

Fedora Service Framework(2005-2006)

F e d o ra Re p o sito rySe rv ice

Serv ices

Apps

P re se rva tionInte grityS e rvice

Ex te rna lW orkflow

JHOV E

GDFR

Ba sicW orkflowS e rvice

Dialog Box Name

O KTex t:

Tex t

Tex t

Tex t

Tex t

Tex t

Canc el

H elp

Sample Text Here Sample Text Here Sample TextHere Sample Text Here Sample Text Here SampleText Here Sample Text Here Sample Text HereSample Text Here Sample Text Here

S am ple Tex t Here S am ple Tex t Here S am ple Tex t Here Sam ple Tex t HereS am ple Tex t Here S am ple Tex t Here S am ple Tex t Here Sam ple Tex t HereS am ple Tex t Here S am ple Tex t Here S am ple Tex t Here Sam ple Tex t Here

Fedora-Web-IRAdministrator

OAIP rovide rS e rvice

Dire ctoryInge st

S e rvice

W e b-ba se dsubm ission a ndba sic w orkflow

Dyna m icDisse m ina tor

S e rvice

PolicyBuilder

Fe doraS e a rchS e rvice

Other

Ser v ice

Fedora Service Framework(2006-2007)

Serv ices

Apps

P re se rva tionInte grityS e rvice

Ex te rna lW orkflow

JHOV E

GDFR

Ba sicW orkflowS e rvice

Dialog Box Name

O KTex t:

Tex t

Tex t

Tex t

Tex t

Tex t

Canc el

H elp

Sample Text Here Sample Text Here Sample TextHere Sample Text Here Sample Text Here SampleText Here Sample Text Here Sample Text HereSample Text Here Sample Text Here

S am ple Tex t Here S am ple Tex t Here S am ple Tex t Here Sam ple Tex t HereS am ple Tex t Here S am ple Tex t Here S am ple Tex t Here Sam ple Tex t HereS am ple Tex t Here S am ple Tex t Here S am ple Tex t Here Sam ple Tex t Here

Fedora-Web-IRAdministrator

OAIP rovide rS e rvice

Dire ctoryInge st

S e rvice

W e b-ba se dsubm ission a ndba sic w orkflow

Pre s e rvationM onitor ing

Se rvice

Eve ntNotification

Se rvice

Fe doraS e a rchS e rvice

Dyna m icDisse m ina tor

S e rvice

PolicyBuilder

Other

Ser v ice

FedoraRepos itory

S erv ice

Fe dora Re po sito rySe rv ice

Fe de rationPID Re s olution

Se rvice

FedoraR epository

Serv ice

You asked…

• “We wish for a out-of-box” end-user client for Fedora.”

• “Can’t you put the DSpace interface on top of a Fedora repository?”

• “We need something to show people Fedora right away (before we get $$ for development resources).”

• “We love Fedora. It would be really great if you distributed a default end-user client.”

The Answer: FedoraWebIR Client

• Web-based client for “institutional repository”• Configurable• End-user submission• Object creation template for “content

models” • Basic Workflow• Search/Browse• Easy configuration of access policy

• Development to begin this summer.

More Dev-Team Priorities

• Federated Repositories• Federation with other repositories (DSpace, aDORE)

– note the Cornell/LANL NSF Pathways project.

• Fedora Showcase and News (on new website)• “Content Model” Specification Language• Advanced Object Creation Workbenches• Tools for RDF browse and graph traversal• Performance Tune – millions of objects• Web services security and Shibboleth• Code Refactoring

• Fedora as web app (.war)• MVC2 pattern for REST-based web exposures• Other misc.

Preview Dinner Talk

• Fedora Development Consortium– Advisory Board– Commission Working Groups– Prioritize Development

• Sustainability Models• Collaborative Development• User-contributed Tools, Apps,

Services

New Fedora Web Site!

www.fedora.info

Solution: Information Network Overlay

DataStores

DocumentRepositories

Databases

WebResources

PublisherRepositories

Information Network API

Source Layer

NetworkRepresentation

Layer

Client Layer

Process-orientation - workflows

Validatebyte-

streams

Ingestto

Repo

Link to Simulation

Service

AssignAccessPolicy

Indexand

Register

Ingest-oriented process

VisitThe

Doctor

FormatMigration

ObjectVersioning

In Repo

MakeCopies

IngestTo

ArchivePreservation-oriented process

IngestTo

Archive

SIP

DigitalObject

World of Services