Post on 14-Dec-2015
B2: Storage and Compatibility
John Knight
Blackboard, Inc.
And now a word from our lawyers.
Any statements in this presentation about future expectations, plans and prospects for the Company, including statements about the Company, the Building Blocks Program and other statements containing the words “believes,” “anticipates,” “plans,” “expects,” “will,” and similar expressions, constitute forward-looking statements within the meaning of The Private Securities Litigation Reform Act of 1995. Actual results may differ materially from those indicated by such forward-looking statements as a result of various important factors, including: product development, and other factors discussed in our Registration Statement filed on Form S-1 with the SEC. In addition, the forward-looking statements included in this press release represent the Company’s views as of July 26, 2004. The Company anticipates that subsequent events and developments will cause the Company’s views to change. However, while the Company may elect to update these forward-looking statements at some point in the future, the Company specifically disclaims any obligation to do so. These forward-looking statements should not be relied upon as representing the Company’s views as of any date subsequent to July 26, 2004.
Topics
Overview Limitations Solutions
– Portability– Structured data
B2 Storage Overview
Building Block data types– Private data
• Configuration data• Operational data (encryption keys, etc.)
– Content data– Tool data – Extended attributes
B2 Storage Problems
Private data– Data that has no relationship to Blackboard entities
Custom attributes attached to Blackboard objects Reference data
– References to Blackboard objects– Custom attributes referencing an external object
Private B2 Storage
Stored in a directory specific to the Building Block– E.g., configuration data– Not visible to other Building Blocks– Data not related to Blackboard entities
blackboard.platform.plugin.PlugInConfig
Accessing Private B2 Storage
PlugInConfig pinConfig = new PlugInConfig( VID, HANDLE );
File configDir = pinConfig.getConfigDirectory();
File configFile = new File( configDir, “configuration.properties” );
Extended Data
User and Course support arbitrary name/value pairs, called “registries”
Package blackboard.data.registry Loader/persister entities Values are portable (i.e., moved in Content Exchange
and copy)– Copy settings may affect the outcome
Values are “public” – visible to other Building Blocks
Extended User Data
User user = ctx.getUser();
//load user registryUserRegistryEntry entry = UserRegistryEntryDbLoader .Default .getInstance() .loadByKeyAndUserId( “key”, user.getId() );
String value = entry.getValue();entry.setValue( “value” );
//store registry valueentry.persist();
Extended Course Data
Course course = ctx.getCourse();
//load user registryCourseRegistryEntry entry = CourseRegistryEntryDbLoader .Default .getInstance() .loadByKeyAndCourseId( “key”, course.getId() );
String value = entry.getValue();entry.setValue( “value” );
//store registry valueentry.persist();
Content Storage
Centralized storage for the bulk of the data that makes up a course
Organized by course, then content Database component
– Single row, fixed fields– Hierarchical
File system component– Private area per content item
Content Storage – B2 View
No “registry” of arbitrary attributes No indexed, semi-structured storage
(e.g., XML data type) File system “blindly” processed by Content Exchange,
Course Copy– E.g., SCORM support is implemented as a Building Block. The
manifest, etc., are stored in the Content item’s private directory.
– “Attached” files also get a database record and are “rendered” by the content list page
– Not all files are “attached”
Content Storage
Database Record
“Attached” File
Content Storage
Attached files are files with ContentFile records in the database
To display a “private” file, a ContentFile record is not required– Just the URL– Useful for creating data files that are referenced by client-side
viewers (e.g., applets)
Content Storage – CX and Copy
By storing “custom” data in the file system component, the data will get picked up by Content Exchange and Course Copy
Data moved as-is, with no transformations As a rule, avoid hard-coding references
– Use template variables
Template Variables
Portable placeholders for referencing Blackboard object data
Dynamically expand at render time, using Context information
No copy/export processing required
Template Syntax
@X@object.attribute@X@ The object/attribute names are taken from the Perl
syntax, since that’s where the bulk of the variable expansion originally occurred.
Referencing an Uploaded File
For example:– <img src=“@X@content.url@X@/uploaded.doc”>
Expands to:– <img src=
“/courses/1/BOB/content/_10_1/uploaded.doc”>
Must Have Templates
content.url – expands to private directory for a particular content item
course.url – expands to root course directory user.user_id – expands to the login name for the
current user course.course_id – expands to the unique, string
identifier for the current course
Template Expansion
Or expanded programmatically…
BbSession sess =
BbServiceManager.getSessionManager()
.getSession( request );
String encoded = sess.encodeTemplateUrl( request, “@X@course.url@X@” );
Can be expanded at render time…
<!-- in content body -->
<a href=“@X@content.url@X@/privateFile.xml”>
Tool Storage
No pre-defined storage location– Ad hoc; no central “repository”– Custom per-tool
• Some db storage, some file storage• Example: Discussion Board
Without B2 database extensibility mechanism, nowhere to put tool data
Custom storage required
Storage Options
Property files XML Data Third party storage engines
Property Files
Easy to read; easy to write– Built into the Java platform– User editable as a fallback
Drawbacks– Name/value pairs can be limiting– I18N limitations (hard-coded ISO-8859-1)
Property Files
FileInputStream fis = new FileInputStream( “block.properties” );Properties props = new Properties();props.load( fis );String param = props.get(“plugin.param.show-trace” );
#a propertyplugin.param.show-trace=true
Property File:
Consumer Code:
XML Files
Reasonably Complex Structure Available libraries Additional complexity
– JAXB?– W3 DOM?– SAX?– DOM4J?
Third Party Storage Engines
Arbitrary capabilities and flexibility Some very easy to use Some, not so much Possible gotchas (system permissions, etc.)
Third Party Storage Examples
Sleepycat Software Berkeley DB Java Edition– Works in web application– Requires file system access
Lucene– Text indexing– Unstructured data
Berkeley DB Example
Java implementation of Berkeley DB Name/value pairs, but with primary and secondary
indexes – Create reasonably complex data structures
File-based storage– When written into B2 private storage, no custom permissions
required
Berkeley DB Example
//create environment on first accessEnvironmentConfig envConfig.setAllowCreate( true );
Environment environment = new Environment( plugInConfig.getConfigDirectory(), envConfig );
//create database on first accessDatabaseConfig dbConfig.setAllowCreate( true );
//we want multiple values for each recorddbConfig.setAllowDuplicates( true );
Database db = environment.openDatabase( null, dbId, _dbConfig );
Berkeley DB Example
User user = ctx.getUser();
DatabaseEntry keyEntry = new DatabaseEntry( user.getUserName().getBytes() );
DatabaseEntry valEntry = new DatabaseEntry();
//create new recordvalEntry.setData( value.getBytes() );db.put( null, keyEntry, valEntry );
Outstanding Storage Issues
Data reconciliation on copy/move– Updating an embedded Id– Updating external references
No hooks for integration yet– Course Copy hooks—reference updates– Content Exchange hooks—transformations to and from
packaged data
Migration
For most plugins, not an issue Subset of R5 to R6 data is migrated (content only) Building Blocks required substantial modifications
anyway
Course Copy
Processes registry entries– Course– User
Copies all content files, regardless of whether they’re registered as “attachments”
Identical rules in Content Exchange
Looking Ahead
Course Copy integration– Post-processing handlers
(See System Admin Logs System Tasks Status)– Coarse-grained notification of a course copy event
Content Exchange integration– Transform data dynamically during the
packaging/unpackaging operation
Common Practice
Per-User data, attach via the Registry Keep private data private
– B2 data, store in private file store– Encrypt sensitive registry values (e.g., external system
password), keep the key in B2 file store
Common Practice
Tool data– Use a storage engine for complex data structures– Existing, external databases can be used
• Requires careful investigation of security issues• Difficult to implement in a portable way
Content data– Use portable references where possible
General– Store “reference” data for sanity checks, e.g., store the original
course id with a record. You can check the current course to know if the data has been copied, etc., and take appropriate action
Conclusion
Know the limitations Understand where and how data can be read and
stored (not just by your code) There are good solutions that work within the (current)
confines of the system– Better ones to come!
Thank You
Demos to Follow >