Digital Object Architecture and the Handle System Larry Lannom 20 June 2006 Corporation for National...

Digital Object Architectureand the

Handle System

Larry Lannom20 June 2006

Corporation for National Research Initiativeshttp://www.cnri.reston.va.us/

http://www.handle.net/

Corporation for National Research Initiatives

What is the Problem?

• Managing information in the Net over very long periods of time – e.g. centuries or more

• Dealing with very large amounts of information in the Net over time

• When information, its location(s) and even the underlying systems may change dramatically over time

• Respecting and protecting rights, interests and value


A Meta-level Architecture

• Allows for arbitrary types of information systems• Allows for dynamic formatting and data typing• Can accommodate interoperability between

multiple different information systems• Allows metadata schema to be identified and

typed


Digital Object Architecture: Motivation

• To reformulate the Internet architecture around the notion of uniquely identifiable data structures

• Enabling existing and new types of information to be reliably managed and accessed in the Internet environment over long periods of time

• Providing mechanisms to stimulate innovation, the creation of dynamic new forms of expression and to manifest older forms

• While supporting intellectual property protection, fine-grained access control, and enable well-formed business practices to emerge


Digital Object Architecture: Components

– Digital Objects (DOs)• Structured data, independent of the platform on which it was created• Consisting of “elements” of the form <type,value>• One of which is its unique, persistent identifier

– Resolution of Unique Identifiers• Maps an identifier into “state information” about the DO• Handle System is a general purpose resolution system

– Repositories from which DOs may be accessed• And into which they may be deposited

– Metadata Registries• Repositories that contain general information about DOs• Supports multiple metadata schemes• Can map queries into unique DO specifications (via handles)


What is a Digital Object

• Defined data structure, machine independent• Consisting of a set of elements

– Each of the form <type,value>– One of which is the unique identifier

• Identifiers are known as “Handles”– Format is “prefix/suffix”– Prefix is unique to a naming authority– Suffix can be any string of bits assigned by that authority

• Data structure can be parsed; types can be resolved within the architecture• Associated properties record and transaction record containing metadata

and usage information


Repository Notion

Any Hardware & SoftwareConfiguration

Any Hardware & SoftwareConfiguration

Logical External Interface

RAP

RepositoryAccess Protocol


Repositories & Digital Objects

REPOSITORY

Each DigitalObject has itsown unique & persistent ID

Objects may beReplicated in

Multiple Repositories

Content Providers assign Ids

No theoretical limitson number of DOs

Per Repository

RAP


• Provides basic identifier resolution system for Internet • Logically centralized, but physically distributed and highly

scalable• Enables association of one or more typed values, e.g., IP

address, public key, URL, with each id• Optimized for speed and reliability• Secure resolution with its own PKI as an option• Open, well-defined protocol and data model• Provides infrastructure for application domains, e.g.,

digital libraries & publishing, network mgmt, id mgmt ...

Handle System


Handles Resolve to Typed Data

URL 2 http://a-books.com/….

DLS 9 acme/repository

HS_ADMIN 100 acme.admin/jsmith

XYZ 100111001111012

Handle dataHandle Data typeIndex

10.123/456 URL 1 http://acme.com/….

Handle Resolution

is a collection ofhandle services,each of which consists of one ormore replicated sites,

Site 1Site 1 Site 2Site 2

Site 1Site 1

Site 2Site 2

Site 3Site 3 …... Site nSite n

Client

The Handle System

LHS

LHS LHS

LHSGHR

each of which mayhave one or moreservers.

123.456/abc URL 4 http://www.acme.com/

http://www.ideal.com/8URL

#1#1 #2#2 #n#n#4#4#3#3

#1#1 #2#2

...

Handle Clients

Global HandleRegistry

1. Sends request to Global to resolve 0.NA/10.1000 (naming authority handle for 10.1000)

Client

Request to Client:Resolve hdl:10.1000/1

Handle Clients

Client



2. Global Responds withService Information for 10.1000

Service InformationAcme Local Handle Service

xcccxv xc xc xc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

..

..

..

xcxcxc

..

..

..

xcxcxc

..

..

..

...

xcccxvxccxxccx

xcccxvxccxxccx

xcccxvxccxxccx

Service Information - Acme Local Handle Service

Primary Site

123.45.67.8

Port #

Secondary Site B

Server 1

Server 1

Server 2

Server 3

Server 1

Server 2 123.52.67.9

321.54.678.12

321.54.678.14

762.34.1.1

123.45.67.4

Public Key ...

2641

K03RLQ...

2641

2641

2641

2641

2641

5&M#FG...

F^*JLS...

3E$T%...

A2S4D...

N0L8H7...

...

...

...

...

...

...

IP Address

Secondary Site A

xcccxv xc xc xc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

xcxcxc

..

..

..

xcxcxc

..

..

..

xcxcxc

..

..

..

...

xcccxvxccxxccx

xcccxvxccxxccx

xcccxvxccxxccx

Handle Clients

Handle Clients

3. Client queries Server 3 in Secondary Site A for 10.1000/1Client


#1#1 #2#2

#1#1

#1#1#2#2

#3#3Primary Site

Secondary Site A

Secondary Site B


Acme LocalHandle Service

Handle Clients

4. Server responds with handle data

Client


#1#1 #2#2

#1#1

#1#1#2#2

Primary Site

Secondary Site A

Secondary Site B


#3#3

Acme LocalHandle Service

Handle Clients

LHS

LHS LHS LHS

LHS

LHS

LHS

LHSGHR

LHS

Handle System

Web ClientHandle Administration

Client

ResolveHandle

HTTP Redirect

Proxy/Web Server

HTTP Get

Handle Data

http://hdl.handle.net/123.456/abc

Handle Clients

LHS

LHS LHS LHS

LHS

LHS

LHS

LHSGHR

LHS

Handle System

ClientHandle Administration

Client

hdl:/123.456/abc

ClientPlug-In

Resolve HandleRequest

Handle Data

Handle Clients

LHS

LHS LHS LHS

LHS

LHS

LHS

LHSGHR

LHS

Handle System

WebHandle Administration

Client

Web Server

HTTP

Handle Admin API

Admin Forms

Handle Clients

LHS

LHS LHS LHS

LHS

LHS

LHS

LHSGHR

LHS

Handle System

WebHandle Administration

Client

CustomClient

Handle Clients

LHS

LHS LHS LHS

LHS

LHS

LHS

LHSGHR

LHS

Handle System

Handle Administrationembedded in another

process


processWeb

Handle Clients

LHS

LHS LHS LHS

LHS

LHS

LHS

LHSGHR

LHS

Handle System


process


process

Handle Resolutionembedded in another

process

Handle Resolutionembedded in another

process


Handle System Usage

• Library of Congress

• DTIC (Defense Technical Information Center)

• IDF (International DOI Foundation)– CrossRef (scholarly journal consortium)

– CAL (Copyright Agency Ltd - Australia)

– MEDRA (Multilingual European DOI Registration Agency)

– Nielsen BookData (bibliographic data - ISBN)

– R.R. Bowker (bibliographic data - ISBN)

– Office of Publications of the European Community

– German National Library of Science and Technology

• NTIS (National Technical Information Service)

• DSpace (MIT + HP)

• ADL (DoD Advanced Distributed Learning initiative)

• Assorted Digital Library Projects

• In development: Globus Alliance


• Assigned Prefixes (June 06)– DOI - 1772

– Other - 801

• Handles– DOI - 22+ M

– Other - Additional millions (total per prefix known only to prefix manager; LANL adding 600M but privately)

• Global– Core: three service sites (added locations being considered)

– 53 M resolutions

Handle System Usage


• Specification– RFC 3650: Overview

– RFC 3651: Namespace and Service Definition

– RFC 3652: Protocol

• DoDI 1322– Will mandate Handle System use as part of ADL-R

• ISO standards track for DOI

• HSAC - Handle System Advisory Committee– Approx 15 members representing big users

– Goal: evolve to oversee the system

Handle System Management and Standards


• Technological and Organizational Infrastructure– Register the existence and access conditions for Learning Objects

relevant to the DoD ‘Enterprise’– Provide user interface to search the registry

• Integrates existing technologies– Handle System for identification and access– XML for object description and submission– LOM metadata– Repository for metadata object storage and access– Lucene search engine

• Running at CNRI in initial production phase

ADL Registry (ADL-R)

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4

Marines

M1

M10

M20

ContentObjects

ADL-R Input

Collections Input Processing Registry

ParseAuthenticateValidateReturn

1N1

metadata

SearchEngine

MetadataObjects

ADL-R

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4

ADL-R Input



N1

metadata

<xml><title>Course 1</title><org>J-School</org><hdl>123/4</hdl>........</xml>

SearchEngine

MetadataObjects

ADL-R

DTIC

LOC IDF NSDL

LHS

UWisc

LHS

ADL-RGHR

LHS

Handle System

NAVAIR has HandlePrefix 123 and names N1 hdl:123/4

Marines

M1

M10

M20

ContentObjects

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4

ADL-R Input



2 ResultsLog

SearchEngine

MetadataObjects

ADL-R

Marines

M1

M10

M20

ContentObjects

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4

ADL-R Input



SearchEngine

MetadataObjects

ADL-R

MetadataObject

matching ContentObject N1

hdl:abc/d

xml

DTIC

LOC IDF NSDL

LHS

UWisc

LHS

ADL-RGHR

LHS

Handle System

Input process createsMetadata Object forN1 named hdl:abc/d...

3

...and creates twohandles: hdl:abc/dfor the MetadataObject & hdl:123/4for the ContentObject.

4

Marines

M1

M10

M20

ContentObjects

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4

Searching the Registry

SearchEngine

MetadataObjects

ADL-R

MetadataObject


hdl:abc/d

xml

Client

Client does a search. Resultspoint to Metadata Object abc/d.

1

DTIC

LOC IDF NSDL

LHS

UWisc

LHS

ADL-RGHR

LHS

Handle System

Marines

M1

M10

M20

ContentObjects

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4


SearchEngine

MetadataObjects

ADL-R

MetadataObject


hdl:abc/d

xml

If desired, client gets Metadata Object abc/d to view full registrymetadata.

2

DTIC

LOC IDF NSDL

LHS

UWisc

LHS

ADL-RGHR

LHS

Handle System


1

Marines

M1

M10

M20

ContentObjects

Client

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4


SearchEngine

MetadataObjects

ADL-R

MetadataObject


hdl:abc/d

xml

Client decides to get ContentObject N1 and resolves handle123/4 to get its accesslocation and other conditions.

3

DTIC

LOC IDF NSDL

LHS

UWisc

LHS

ADL-RGHR

LHS

Handle System


1


2

Marines

M1

M10

M20

ContentObjects

Client

ATSC

A3 A3 A2 A2

A1A1

ContentObjects

NAVAIR

N4N4

N3N3

N2N2

ContentObjects

N1

hdl:123/4


SearchEngine

MetadataObjects

ADL-R

MetadataObject


hdl:abc/d

xml

Client requests a copy ofContent Object N1 from NAVAIR.

4

Client decides to get ContentObject N1 and resolves handle123/4 to get its accesslocation and other conditions.

3


1


2

DTIC

LOC IDF NSDL

LHS

UWisc

LHS

ADL-RGHR

LHS

Handle System

Marines

M1

M10

M20

ContentObjects

Client


CORDRACommunity

CORDRARegistry

ContentRepository

Object LevelMetadata

ContentRepository

ContentRepository

ContentRepository

ADL-R

ContentRepositories

CORDRACommunity

CORDRARegistry

CORDRARegistry

Community

ContentRepositories

CORDRACommunity

ContentRepositories

CORDRARegistry

IntermediateRegistry

of Registries

FederationLevel

Metadata

CORDRARegistry

CORDRACommunity

Federation Level

Metadata

FederationLevel

Metadata

CORDRACommunity

CORDRARegistry

ContentRepositories

CORDRACommunity

CORDRARegistry

ContentRepositories

IntermediateRegistry

of Registries

CORDRARegistry

Community

Federation LevelMetadata

MasterRegistry

of Registries

CORDRARegistry

Community

ContentRepositories

CORDRARegistry

Federation LevelMetadata

CORDRA

Digital Object Architecture and the Handle System Larry Lannom 20 June 2006 Corporation for National...

Documents

Transcript of Digital Object Architecture and the Handle System Larry Lannom 20 June 2006 Corporation for National...