Voyager Meets MeLCat: MC'ing the Introductions
-
Upload
roy-zimmer -
Category
Technology
-
view
526 -
download
4
description
Transcript of Voyager Meets MeLCat: MC'ing the Introductions
Voyage meets MeLCat:MC’ing the Introductions
MeLCat extract sequences
Voyager
bibout.pl
bib extract
patout.pl
today’s extract
yesterday’s extract
patdiff.pl
Pmarcexport
patron extractto MeLCat
Our MeLCat implementation is unfortunately still a work in progress.
The information presented here may be subject to change.
The information is as current as possible, given deadline constraints.
Disclaimer
The bib extract and patron extract programs, and the patron differences program, are all written in Perl and are designed to run on your Voyager box.
A Unix box is assumed. Minimal modifications should be required in order to run on a Windows box.
Notes
User requirements:
Familiarity with a text editor, file names, and paths on your machine.
You’ll need to be able to query your database (SQL provided).
You will be editing the setup section at the beginning of the programs to customize settings for your institution.
Notes
MeLCat extract sequences
Voyager
bibout.pl
bib extract
patout.pl
today’s extract
yesterday’s extract
patdiff.pl
Pmarcexport
patron extractto MeLCat
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
Bib Extract, Pmarcexport
The export command is in script form in a file. Thus the parts of the command have been broken out. The trailing “\” indicates the command continues on the next line.
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
Bib Extract, Pmarcexport
Substitute your own values for words in upper case.
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
Bib Extract, Pmarcexport
The mode, m, is B, which means to select records by both create and update dates.
The export target, t, gives the date range.
The last parameter, i, says to ignore suppressed bib records.
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
Bib Extract, Pmarcexport
today-N specifies to look back N days from today
At WMU, we used a value of 10,000 for our initial load. For ongoing loads, we use a value of 1, since we run our MeLCat bib extract daily.
Bib Extract, Pmarcexport problems
We had problems with Pmarcexport.
Our initial large bib export resulted in two files, not one!
This turned out to be expected, but previously undocumented, behavior.
Bib Extract, Pmarcexport problems
We had problems with Pmarcexport.
Our initial large bib export resulted in two files, not one!
This turned out to be expected, but previously undocumented, behavior.
Had problems with record counts when a split occurred.
Bib Extract, Pmarcexport problems
We had problems with Pmarcexport.
Our initial large bib export resulted in two files, not one!
This turned out to be expected, but previously undocumented, behavior.
Had problems with record counts when a split occurred.
Also had mysterious SQL error which didn’t(?) seem to affect the output.
When we finalized this presentation, these issues had not yet all been resolved.
MeLCat extract sequences
Voyager
bibout.pl
bib extract
patout.pl
today’s extract
yesterday’s extract
patdiff.pl
Pmarcexport
patron extractto MeLCat
bibout.pl voy_bib_export_file melcat_bib_file
voy_bib_export_file is created by Voyager’s Pmarcexport program
melcat_bib_file is the resulting bib file ready to be sent to MeLCat
Both files are in MARC format.
Bib Extract, Usage
Typically, Pmarcexport will create one output file. For large amounts of data, you may have two or more files.
In that case, you can run bibout on each file, or possibly concatenate the Pmarcexport files into one for bibout to process.
In this presentation, and in bibout.pl, Pmarcexport’s output is referred to as one file.
Bib Extract, Usage
Two stages:
test extracts and initial loads
ongoing production extracts and loads
The exact usage is determined via a Pmarcexport script parameter and the corresponding setting for bibout.
Bib Extract, Usage
Bib Extract – What does it do?
bibout reads the file created by Pmarcexport.
For each bib, find and scan its mfhd(s) for fields 866, 867, and 868. These fields, if present, are added as is to the bib record.
Get mfhd and item information associated with this bib record, or just mfhd data as a limited fallback when there is no item record.
Bib Extract – What does it do?
Append a new field to the bib record that contains the following data:
|a item id (mfhd id if no item) |b item barcode |c item status |d item type |e call number |f location |g enum info |h number of recalls
This is field 999 by default.
Bib Extract – What does it do?
|a item id
When there is no item id, |a contains a mfhd id. This is indicated with a leading “M” in this field.
Still to come: handling deleted records
Bib Extract, database access
bibout needs to know how to connect to your Voyager database.
At WMU, we’ve standardized on storing this information in a file, allowing for easier updates and better security. Any programs that access Voyager do not have the specific database access information in the program.
Bib Extract, database access
The information you need for this is your:
machine name
database name (may be username)
database login (password)
Oracle SID [VGER | LIBR]
Voyager 5’s SID is VGER; the SID for previous versions is LIBR. Use the right one for your version.
Bib Extract, database access
Put this information in the file, all on one line:
your.machine.name.edu/namedb/db_login/sid
(namedb may be username in your situation)
The slashes, “/”, belong in there, and the bib extract program will read this data so it can access Voyager. Make sure that this file contains your information, including the SID.
Then we need to tell bibout where this file is, by specifying its path and the file name.
Bib Extract - which database?
bibout needs to know from which database we are extracting the data.
Since the database name may not equate to the user, we specify the database name separately.
For example, you may be connecting to your database as a read-only user, so you would have to specify the database.
Bib Extract
bibout also needs to know how many days to look back.
Give it the same value, N days, that you used in the export script.
This makes sure that everyone is on the same page.
Bib Extract – Which locations?
We wanted to restrict, by location, the items showing up in MeLCat.
Here’s a sample query to see your location codes:
Select the location_codes you want, and put them in a text file, one per line.
Bib Extract – Which locations?
After creating the location codes file, you need to tell bibout where this file is (the path), and the file name.
Bib Extract
By default, the new field that contains the item/mfhd information is 999.
If you need a different tag, you’ll need to tell bibout what that is.
MeLCat extract sequences
Voyager
bibout.pl
bib extract
patout.pl
today’s extract
yesterday’s extract
patdiff.pl
Pmarcexport
patron extractto MeLCat
Patron Extract, Usage
patout.pl patron_extract_file
patron_extract_file is the double quoted, comma separated value format patron data retrieved from Voyager
Patron Extract – What does it do?
Similar to bibout, the patron extract program (patout.pl) also looks back so many days to get patron data.
Voyager is queried and the patron extract data for MeLCat is put in CSV format.
Patron Extract, Output
last name, first name, middle name,
patron barcode, patron group, patron exp date,
WIN (WMU ID number), patron ID (Voyager internal number),
delete indicator (D=delete, blank to retain), blocked (Y/N),
email address, email exp date
patron exp date is 2382.12.31, if indefinite
Patron Extract, blocking
Patron is blocked when:
patron record expired before today
lost items count exceeds lost limit defined in the extract
recalls count exceeds recall limit defined in the extract
fees due amount exceeds the maximum balance for the patron group used in the extract
email address record expired before today
Patron Extract, assumptions
The extract uses the patron’s most recent active patron group as the patron group of record.
Patron Extract, database access
The patron extract gets its Voyager database access information from the same file mentioned earlier in the bib extract part of this presentation…
Database Access
Put this information in the file, all on one line:
your.machine.name.edu/namedb/db_login/sid
(namedb may be username in your situation)
The slashes, “/”, belong in there, and the bib extract program will read this data so it can access Voyager.
…then we need to tell patout where this file is, by specifying its path and the file name.
Patron Extract
Again, like with bibout, we need to tell patout how many days back to look to get patron data.
This value will probably depend on how often your patron data is updated.
Patron Extract
We discovered setting up limits by patron group to be pretty much impossible. This is because a patron group can be in multiple circ groups, with each circ group able to have different limits.
So we went with across-the-board limits, and you’ll have to use a global value for the lost-limit and the recall-limit.
Patron Extract, patron groups
There will probably be some patron groups that should not be in MeLCat.
You’ll need to tell patout which patron groups are allowed. This is done via the patron group ID.
Here’s how to get these ID numbers:
Patron Extract, patron groups
Now you have a listing of patron group IDs and names. Note that fine/fee limits can be found in the Systems Administration client by clicking on Circulation, then on Policy Definitions.
Record your patron group limits at each corresponding patron group ID.
Remember that in Voyager, money values (U.S.) are stored in cents, so the values you enter here, divided by 100, equal the dollar amounts.
Patron Extract
We also need a representative circ group for patout to do its thing.
From your Sys…Admin… client, pick a circulation policy group with the most appropriate lending attributes. Now run a query to get its ID number:
and let patout know what it is.
Patron Extract
Finally, patout needs to know where to write the output. Specify the file path and file name.
The filename will automatically have a date stamp appended.
For example, if you’re using a file name of “patron.out”, you’ll end up with file names like patron.out.20060825
patron.out.20060826
etc.
MeLCat extract sequences
Voyager
bibout.pl
bib extract
patout.pl
today’s extract
yesterday’s extract
patdiff.pl
Pmarcexport
patron extractto MeLCat
Patron Differences Extract
patdiff.pl looks at the current and the previous patron extract and puts the differences into the patron extract to be sent to MeLCat.
The extract comparison is based on patron ID.
Patron Differences Extract
The extract gets:
new records
records no longer showing up; they aremarked for deletion
the current version of any records with differences in them
Patron Differences Extract
patdiff needs to know where to write its output file. Specify the file path and the file name.
Extracts ready!
Your bib and patron extracts are now ready to be sent to MeLCat.