DigiTool Database Structure DigiTool Version 3.0.
-
date post
19-Dec-2015 -
Category
Documents
-
view
243 -
download
1
Transcript of DigiTool Database Structure DigiTool Version 3.0.
DigiTool Database Structure
DigiTool Version 3.0
DigiTool System Architecture2
Session Agenda
Overview and Terms
Directory Tree structure
Databases/Units
DigiTool System Architecture3
Meditor client = Staff module (tool) for editing and management of metadata, objects and users.
Web-based Resource Discovery = Web interface (tool) for end-user collection searching, viewing and use.
Repository = Where DigiTool resources are stored.
Deposit Module = Web interface (tool) for end-user material deposits and staff review.
Ingest Module = Web interface (tool) for staff loading of material.
Collection Management Module= Web interface (tool) for managing hierarchical collections.
Staff Management Module= Web interface (tool) for staff management of repository, delivery and deposit.
DigiTool Terms - General
DigiTool System Architecture4
Session Agenda
Overview and Terms
Directory Tree structure
Databases/Units
DigiTool System Architecture5
Databases/Units
Three types of units exist:
2. Silo Unit is a database that holds the data and configuration for the Web-based Resource Discovery e.g. GEN01
1. Admin Unit is a database that controls the work activity of cataloging, deposit, ingest, etc. for that specific unit only. e.g. DTL01, DTL02
3. System Unit is a database that controls system-related activities such as users, repository, storage and cache e.g. DAT01, REP00, EXT01, VIR01
DigiTool System Architecture6
Session Agenda
Overview and Terms
Directory Tree structure
Databases/Units
DigiTool System Architecture7
Overview
Two main types of directories exist:
2. Global directories such as dtle or system
include global configuration files used across local units/databases.
1. Local directories of databases (units) contain files/tables with definitions that are local to specific databases (e.g., dtl01, gen01, etc.).
DigiTool System Architecture8
Overview II
Each directory contains sub-directories, which include configuration files and tables.
There are aliases (shortcuts) for some sub-directories (e.g., dt for local unit’s tab/ ), which will be noted in italicized font throughout this presentation.
DigiTool System Architecture9
DigiTool Directory Structure
exlibris
aleph dtlsfx metalib verde
d3_1 j3_1 u3_1Install - Default
DigiTool System Architecture10
The d tree– DigiTool Default
Sub-directories
Directories
Version
dtl
d3_1
dtlm
dtl pds dtl01
dtl02
vir01 tmp log
dtle*
DigiTool System Architecture11
d3_1 directory & aliases
d3_1dtlm
dtl
vir01
dtl01 & dtl02
tmp
log
pds
xyz_media
service pack
$TMPDIR
$LOGDIR
dm, dpm
$dtl_dev
Software - development & maintenance programs
Software executables, scripts
Temp system unit
Demo admin units
Server logs
Service Pack installations
dp, de
Temp Media – web use
Temporary directory
Patron Directory Services
DigiTool System Architecture12
The u tree– DigiTool User
Sub-directories
Directories
Version
dtl
u3_1
dtle rep00 xxx01 yyy01 zzz01 ext01dat01gen01
DigiTool System Architecture13
u3_1 directory
Node management – global configuration - active
Local users unit
Local silo unit
Local admin units
u3_1
dtle
dat01
gen01
ext01
xxx01
yyy01
Local external unit
$dtle_root
DigiTool System Architecture14
The j tree– DigiTool
Sub-directories
Directories
Version
dtl
j3_1
profile
digitool
home
system
work
DigiTool System Architecture15
System configuration
Local configuration
Work and temp directory
j3_1
system
profile
work
j3_1/digitool/home/ directory
j_home
DigiTool System Architecture16
• Each unit has a separate root directory.
• Each unit contains information relevant to the configuration of each unit defined in the system.
• Each unit is implemented as its own ORACLE user.
Unit Structure
DigiTool System Architecture17
The Tree Structure - Unit
Sub-directories
Directories
Version
Sub-directories
Version
dtl
u3_1
dtle rep00 yyy01 zzz01… ext01dat01gen01xxx01
DigiTool System Architecture18
The Local d/u Unit Root Directory
To get to the root directory of the collection you are in, type “dr” . dr functions based on the dlib environment which you are located.
dtl01/ - is the $data_root of the dtl01 unit tree/database.
DigiTool System Architecture19
Unit Root – d/u tree
xxx01
tab/
files/
print/
md/
pers_coll_forms/
scratch/
form_eng/
object_xyz/
dr
prof_library
file_list
dpr
df1
dt
dmd
ds
Silo only
DigiTool System Architecture20
Local Sub-directories I
tab/ includes configuration tables for defining the unit’s behavior. (dt or cd $data_tab)
md/ includes tables for defining the unit‘s behavior during Meditor metadata cataloging. (dmd or cd
$data_md )
files/ Dump files (znn.seqaa) and batch queue management (df1or cd $data_files )
print/ includes printouts. It can be accessed using the Meditor’s Task Manager. (dp or cd $data_print)
L o ca l D ire c to ry
DigiTool System Architecture21
Local Sub-directories II
form_eng/ includes email forms, templates for printing, batch services and reports (e.g., registration approval)
pers_coll_forms/ includes forms for the wizard steps for adding personal objects to the e-Shelf from the Resource Discovery.
scratch/ includes temporary files and unit-specific log files. (ds, cd $data_ scratch)
L o ca l D ire c to ry
DigiTool System Architecture22
Local Sub-directories III (Silo ONLY)
object_view/ includes harvested XML files that may be used for “Search in XML” advanced search from the RD.
object_thumbnail/ includes any thumbnail harvested into the silo and for use in the RD.
object_index/ includes all full text indexing files that are harvested into the silo.
object_ctx/ includes all indexed data (full text and otherwise) that is loaded into the DB. Indexes can be built based on the data here or from scratch.
L o ca l D ire c to ry
DigiTool System Architecture23
Local Filesfile_list defines all the Oracle tables required for that
unit, their maximum sizes and their indexes.
L o ca l D ire c to ry TAB z52 10K 10K ts0 IND z52_id 10K 10K ts1 TAB z00 2M 1M ts0 IND z00_id 100K 100K ts1 TAB z01 2M 1M ts0 TAB z00h 100K 100K ts0 IND z00h_id 100K 100K ts1 TAB z101 400K 100K ts0 IND z101_id 100K 100K ts1 TAB z104 100K 100K ts0 IND z104_id 100K 100K ts1 IND z104_id1 100k 100k ts1 TAB z124 400K 100K ts0 IND z124_id 100K 100K ts1 IND z124_id2 100K 100K ts1
DigiTool System Architecture24
Unit Root – j tree
jdd
jdl
jdc
jdrxxx01
conf
load
deposit
DigiTool System Architecture25
Local Sub-directories I
L o ca l D ire c to ry
conf/ Unit configuration including access rights, task chains and template settings. (jdc or jdata_conf)
deposit/ Unit deposits are stored in this directory until staff approval or deletion.
(jdd or jdata_deposit)
load/ Unit-owned ingests originate from this file system location. (jdl or jdata_load)
DigiTool System Architecture26
Navigating – Unit
root directory of local database dr cd $data_root/(u3_1/xxx01/)
jdr cd $jdata_root
(j3_1/digitool/home/profile/units/DTLXX/)
Unit = databases (dtl01, dtl02, gen01, etc.)
metadata directory of local database
dmd cd $data_md
(u3_1/xxx01/md/)
tab directory of local database
dt cd $data_tab
(u3_1/xxx01/tab)
scratch directory of local database
ds cd $data_scratch
(u3_1/xxx01/scratch/)
jdc cd $jdata_root/conf
jdd cd $jdata_root/deposit
jdl cd $jdata_root/load
j tree
d/u tree admin unitd/u tree
DigiTool System Architecture27
The Tree Structure - dtle
Sub-directories
Directories
Version
dtl
u3_1 d3_1
dtlm
dtle rep00 dtl01
dtl02
gen01
dat01
vir01
DigiTool System Architecture28
dtletab/
scratch/
error_lng/
www_r_lng/
gate/
pc_b_lng/
coll_mng/
apache/
dtle Sub-directories
Directory
Sub-directories
apc
pcb
wr
DigiTool System Architecture29
dtle Sub-directories I
The dtle directory includes files that define behavior on the node level, or across unit databases.
Some files that sit under the root (base) directory (cd $dtle_root): e.g.
dtl_start dtl_start.private dtl_shutdown_all dtl_startup_all pc_server_defaults www_server.conf
d tle
DigiTool System Architecture30
dtle Sub-directories II
tab/ includes tables for defining active bases, logical-bases and general configuration such as Resource Discovery behavior. (cd $dtle_tab).
Some important files are:
tab_base.engwww_r_conf.xmltab_dc_to_marc
d tle
DigiTool System Architecture31
dtle Sub-directories III
scratch/ includes log files (detailed files with information on the programs that run in all units). (cd $dtle_scratch)
www_r_eng/ includes HTML and .css files for the Web-based Resource Discovery.
(alias is wr)
d tle
DigiTool System Architecture32
dtle Sub-directories IV
apache/ defines apache settings in regards to DigiTool Resource Discovery.
(alias apc)
error_lng/ includes error and information messages displayed online.
coll_mng/ includes configuration and relevant files for collection management
d tle
DigiTool System Architecture33
dtle Sub-directories V
gate/ z39.50 Universal Gateway configuration.
pc_b_lng/ Meditor services related configuration.
(alias pcb)
d tle
DigiTool System Architecture34
System configuration
Local configuration
Work and temp directory
j3_1
system/
profile/
work/
j3_1/digitool/home/ profile directory
j_home
Note: j3_1/digitool/home.orig is used for service pack mechanism comparisons and should not be moved or changed.
DigiTool System Architecture35
profile
conf/
reports/
export/
units/
collection/
overwrites/
work/
J-tree profile global directories
Directory
Sub-directories
global.properties
General local configuration
Export of digital entities
Admin units
Collection staging location
ALL system/conf changes made in this directory.
Work and temp directory
Repository report output
DigiTool System Architecture36
Profile Sub-directories I
conf/ Global configuration across admin units including copyrights.
reports/ Output of Repository-related services.
export/ Repository exports output in this directory.
profile
DigiTool System Architecture37
Profile Sub-directories II
collection/ Collections and related objects marked for export are stored here until p-media-35 service run.
overwrites/ Any home/system changes are made here and implemented by running set_globals.sh
Note: .tmpl files are used for specific changes to repository_configuration.xml and conn_conf.xml.
work/ Index and temporary file staging area.
units/ All active admin unit directories are stored here.
profile
DigiTool System Architecture38
Profile root Configuration File
global.properties – general properties of the DigiTool version
profile
dbconnection.url=jdbc:oracle:thin:@server:1521:dtl99
dbconnection.username=d31_rep00
dbconnection.password=d31_rep00
server.name=server
http.port=1801
jndi.port=2801
rmi.port=3801
rmi.object.port=3901
webservice.port=4801
server.pool.port=6801
pds.port=8881
pds.server=server
index.directory=/exlibris/dtl/j3_1/digitool/home/profile/work/indexes/d31_rep00
oracle.index.directory=/exlibris/dtl/j3_1/digitool/home/profile/work/indexes/d31_rep00
index.files.per.directory=1000
DigiTool System Architecture39
Active System configuration
Local configuration
Work and temp directory
j3_1
system/
profile/
work/
j3_1/digitool/home/system directory
j_home
j_work
j_sys
DigiTool System Architecture40
systembin/
client/conf/sql/
thirdparty/xsl/
J-tree system global directories
Scripts and programs
General system config
SQL schema
TP applications and web log
XSL schema and mapping
Java client
j_sys
j_bin
j_conf
Directory
Sub-directories
Note: j3_1/digitool/home/system is the active system, but should not be changed directly. Any changes to the system are placed in the profile/overwrites directory and initiated by running the script (system/bin) set_globals.sh.
DigiTool System Architecture41
System Sub-directories I
bin/ Scripts and programs. (e.g. starting and shutting down JBOSS)
(j_bin)
conf/ Global configuration e.g. repository, ingest, e-Shelf, internationalization. (j_conf)
xsl/ XSL configuration and mappings.
system
DigiTool System Architecture42
System Sub-directories II
sql/ SQL schema for J-tree
thirdparty/ Third-party tools including open server and log files.
(jb_log – server.log)
client/ Java system clients.
system
DigiTool System Architecture43
Navigating – Global directories -Summary
$dtle_root (dtle) = root directory of dtle (u3_1/dtle)
$dtle_tab = tab directory of dtle (u3_1/dtle/tab)j_home cd $jdtlhj_bin cd $jdtlh_binj_conf cd $jdtlh_confj_sys cd $jdtlh_systemjb_log cd $jdtl_jb_def_log
DigiTool System Architecture44
Log Files and Monitoring Servers
Web Server Log>>cd $LOGDIR
www_server_4881.logUTIL-W
PC Server Log>>cd $LOGDIR
pc_server_6881.logUTIL-W
comlog – soft links to the log directories
/exlibris/dtl/logs/log_3_00_1
JBOSS Application Server Log
>> jb_log server.log
Apache Server Log>> apcl
access_log
DigiTool System Architecture45
Important Server CommandsRe-start Web Server
>>start_w
Re-start PC Server>>start_pc
Re-start JBOSS Application Server>> j_bin >> jboss_shutdown.sh>> jboss_startup.sh
Re-start all DigiTool Servers >> dtle
>>dtl_shutdown_all >>dtl_startup_all
Re-start Apache>> apcb>>./apachectl restart (or ./apachectls_auto for SSL)
DigiTool System Architecture46
www.exlibrisgroup.com
Thank you!