IBM Lotus Domino 7 and DB2 - Effective Use of Access and...
Transcript of IBM Lotus Domino 7 and DB2 - Effective Use of Access and...
-
BP307: IBM Lotus Domino 7 and DB2Effective Use of Access and Query Views
Dr. Ludwig W. NastanskyIngo Erdmann
-
Dr. Ludwig W. Nastanskyprofessor of business information systems
head of Groupware Competence Center (GCC)
in Lotus Notes and Domino research & business since Release 1
founder of several university spin-off companies
co-founder of German Notes User Group (DNUG)
Ingo Erdmannresearch assistant
9 years of Lotus Notes experience
certified instructor, developer, admin
Speakers
-
Agenda
1 Introduction2 DB2 Access Views (DAV)
scenariolive demo
3 Query Viewsscenariostechnology, tipsfederated data paradigmlive demos
4 Summary
-
1 Introduction
Session focus on query view and integration scenariosGoals
put Lotus Notes and Domino DB2 integration in right contextlearn about innovative efficient integration optionsenvision new approaches for application developmentstart thinking about required changes in infrastructuretake home tips for first implementations
No NSFDB2 basics will be covered
-
Topic for presentation: Specific aspects of NSFDB2The new RDB-functionalities provided by DAV & query viewsRDB functionalities in Notes have been an issue since Release 2Many successful approaches up to now
DECSLEISAP connector
NSFDB2 is the smoothest and most radical integration everBesides that, even the DB2 aficionados will finally admit that Domino is based on a real & powerful database engine …… the core issues are unchanged
integration is not (only) about technologybut (much more) about application domains, IT-strategies, top-down system architectures, functionality concepts
Notes/Domino & DB2 Integration, finally… ?
-
Documents vs. Relational Data
© Delphi Research
Lotus Notes andDomino World DB2/RDB World
-
Lotus Notes and Domino World
Documents vs. Relational Data
strategic orientation & communication centricknowledge & information managementtool paradigm on objects – code re-usecompound documents, semi-structured, very flexible data typesmultimedia, links, embedded methodsdecentralized, buttom-up, user workplace & collaboration centricreplication, information sharing, robust distribution, redundancy, message objectsindex engine optimized for static access to unstructured datasupport of mobile, nomadic and disconnected user workplace
DB2/RDB World
operational orientation & data centrictransaction processing & high volumeautomation paradigm on data – code efficiencyrecords, tables, structured data, restricted flexibility, strict formatstransactions, dynamiccentral organization, top-down, system centric
access coordination, referential integrity, redundancy elimination, 2-phase commit, ACIDindex engine optimized for dynamic access to structured datastatic office-based and server-connected workplace
-
Integration, Cooperation, Federation
Challenge: bring together the best of both worldsas much as needed and as much as makes sense from a business value perspective
Position: Domino RDB integration is just one instance of the innumerable system integration tasks taking place currently
The integration is between equal partnersstay cool, no religion involved, don't demonize
Purposeintegration of data across system boundaries
cooperation of separate applications
federation of Domino applications with DB2-data and, vice versa, DB2 applications with Domino data
NSFDB2 brings RDB/SQL functionalities to Notes and Domino
Let us start with one principal data modelling challenge using a typical scenario
-
Solution in Domino is clumsyWe need to hard-code relational dependancyby duplicating data in documents and thuscreating redundancy- Option #1: Lookup "LastName" in "Employees" view and save in Room-form- Option #2: Lookup "Room" in "Room list" view and save in Employee-form- [Option #3: Enforce consistent user entry in related documents]
#1 #2
Start: twoindependentlists
Goal: modelEmployee-Roomrelation
Scenario 1 - Model Employee–Room Relation
-
Domino challenge: Model Employee–Room Relation
Problem:View "Employee in room" cannot be generated without duplicating "Room"-field and "LastName"-field in "Employee"-document and "Room"-document respectively
reason: @LookUp not allowed in view columns
Synchronization has to be modelled
Modelling the synchronization is possible, but clumsy, e.g.use scheduled agents to collect & update changes
set up (very) disciplined user entry infrastructure enforcing consistent update over involved document collection
Avoiding redundancy of field entries would help considerably
Solution with new options of NSFDB2: Data duplication is not necessary using DB2 Access Views (DAV) and query views
-
Remark
Slides marked with the "mouse" symbolwill be covered during live demo sessions
-
Demo 1: Relations
SQL JOIN to find employee in room
Show room documents
Room documents do not contain the employee name
Use SQL join to find "employee in room"
-
DB2 Access Views and Query Views
NSFDB2
DB2
update, insert, deleteupdate, insert, delete
Domino data flowDAV related data flowControl
Domino
Domino Managed Data
SQLApplications
redundantlystore
Access Views
Access Views
Access Views
Access Views
DAV
read
read
read
DB2Access
DB2 data flowAccess Views
Access Views
form basedselection
user basedselection
DB2 Table
DB2 View
DB2 Managed Data
NotesQueryView
Notes Client
-
Agenda
1 Introduction
2 DB2 Access Views (DAV)scenariolive demo
3 Query Viewsscenariostechnology, tipsfederated data paradigmlive demos
4 Summary
-
DB2 Access Views
-
DAV - Overview
Redundantly store Domino data in DB2 table
DB2 view that corresponds to the table is used for SQL operations
Each row represents one Domino document
DAV is not a Notes UI element, i.e. not Notes standard view
Expose Domino data toSQL applications
JDBC, ODBC
DB2 tools and applications
relational reporting tools (Crystal Reports etc.)
query views
No DECS, LEI, connectors needed
-
DAV – Field Definition
Field 6
DAV DesignDocument
Field 5
Field 4
Field 3
Field 2
Field 1 Fields are elementsof forms
Fields are itemsin notes
New fields, that do notexist in Domino database yet
etc. …
-
DAV – Architecture Summary
Using SQL, you canread Domino data with security semantics enforcedfrom an SQL perspective, this adds "row level security" to DB2 datainsert, update, delete with full Domino semantics
DB2 handles security for read operationsDB2 Access for Lotus Domino handles Domino security on insert, update, delete
DAV calls DB2 Access serverDomino handles replication conflicts, document locking etc.Domino handles ACL, reader fields etc.user mapping required for security(Domino Administrator)
-
Scenario 2: Integration #1
Application and data integration demands for Domino and DB2
analysis using MISreporting using Crystal ReportsHR uses existing DB2 applicationsalesforce uses mobile Notes applicationsalesforce orders are processed by DB2 applicationcustomer uses J2EE application with DB2 backend to file orders
Challengeintegration of applicationssynchronization of data
-
Scenario 2: Integration #2
Solution: integration via DAVmake document based data available for corporate SQL applications via DAV:
expose Domino data to DB2 for read and writepopulate documents with DB2 based data from SQL applications via DAV
make DB2 data available for documentsmake DB2 data available in flexible Notes view context
Benefitsenrich Notes and Domino environment with DB2 based application optionsenrich DB2 environment with Notes and Domino based application options
-
Scenario 2: Mobile Salesforce
Salesforce uses mobile Notes applicationOrders are replicated into central NSFDB2DB2 application works on data exposed via DAV
Notes adds offline functionality to DB2 via replicationTransactions are processed in DB2
Salesforce replicates transaction status back to mobile device
-
Demo 2: DAV Design, Use & Remarks
Updated by DB2 application
Look at DAV elements in DB2views
tables
data
Modify SQL data from Notes
-
DAV Use
-
DAV Remarks
Structure clash between Domino flexibilityand restricted DB2 tables
DB2 requires fixed column/field lengthespecially text fields have to be considereddo not modify truncated data
Think about multi value handlingList fields
reduce DB2 column lengthsum length for multi value fieldsuse alias if possible
-
DAV Summary
Domino Designer as a development tool for DB2Expose Domino data to SQL applicationsMake Notes and Domino functionalities easily available for DB2 applications, examples:
Domino semantics offer additional value to SQL applications("row level security")enrich DB2 with disconnected options
Potential to lower cost for application integration withexisting infrastructure
-
Agenda
1 Introduction2 DB2 Access Views (DAV)
scenariolive demo
3 Query Viewsscenariostechnology, tipsfederated data paradigmlive demos
4 Summary
-
Query Views
-
Query Views - Overview
Notes views which are enabled forSQL queriesSQL queries are used in view selection formula contextUsage examples
filter documents (dynamic view selection formula)add DB2 data to a Notes viewuse retrieved DB2 data in column formulascombine Domino data with external DB2 data
-
Query Views – Data Federation #1
Rows can be populated bydocuments in the current Domino database (via DAV only)
DAV data in DB2 not in the current Domino database
DB2 data
combinations between these via SQL JOIN
Thus query views can display data fromcurrent Domino database exposed by DAV
other Domino databases exposed by DAV
Domino documents exposed by DAV, with additional data via SQL JOIN
Native (non Domino originating) DB2 data
Each row does NOT necessarily represent one document
Keep this in mind during clicking rows
-
Query Views – Data Federation #2
"Federated data" do not originate from current Domino databaseA row can contain document values plus federated data
"normal" Domino field contenthas to be included in DAVhas to be included in SQL selection
result of a column formula calculationDB2 data objects retrieved using SQL Query"double click" yields opening of document
-
Query Views – Data Federation #3
A row can be defined by federated data onlyall values result from DB2 data objects"double click" on row makes no sense (in most cases) and yields error messagecan be used for data consolidation from different NSF files
same applicationdifferent applicationsee "Scenario 5"
-
Query Views – SQL Query Rules
Query is defined in Notes formula language contextQuery supports standard SQL
SQL JOINSQL UNIONORDER BYetc.
Queries that do not produce a result set are not allowedsecurity mechanismprevents deletion/update from view
-
Query Views Are Dynamic
No persistent view index involved
Efficient DB2 indexing is used
Queries can be user specific
Parameterized and personalized
Lookups are allowed to collect Domino data for queryconstruction
Context can be considered when embedded
-
Scenario 3 – Federated Data in Views
HR uses Notes and Domino to manage resource dataEveryone can view employee name and phone numberOnly HR is allowed to view salarySecurity challenge
field encryption requires complex key managementmechanisms such as "hide when" are not security features@DBLookup is not supported in column formulas
Solutionstore data in different database or documentuse query views and SQL JOIN to add federated data
-
NSFDB2 EmpUSA.nsf
DAVEMPLOYEE
NSFDB2 Salary.nsf
DB2
DAVSALARY
EMPID,NAME,PHONE
EMPID,SALARY
Query View D1
NAME PHONE SALARY EMPIDRecord Set 1Record Set 2Record Set n
B
A
SQL JOIN
Result Set of:“SELECT
A.NAME , A.PHONE , B.SALARY, A.EMPIDFROM
EMPUSA.EMPLOYEE A LEFT OUTER JOIN SALARY.SALARY B
ON A.EMPID = B.EMPIDORDER BY
A.NAME”
-
Demo 3
Design a query viewJOIN data from other Domino database
Data from other database
-
Remarks
In most scenarios, LEFT OUTER JOIN is most applicable
ensures all documents are displayedleaves column empty if federated data does not match
Do not include multiple NoteID fields in result setuse specific selection instead of *wrong document might be openedmultiple rows with equal NoteIDs will show as single row
-
Scenario 4 – Dynamic Selection
Workflow application
Very large number of documents
Users only need a small subset of documents
Users want to select which documents to display
Users want profile based personal selection settings
Challenge (in Notes context)@DBLookup in selection formula is not possible
@UserName only works in private views
bad performance due to large view index
Solutionquery views don't have a static view index
query statement(s) can be based on dynamic Notes formula mechanism
-
User selected data
Demo 4
Design a query view
Dynamic selection using @Prompt
Dynamic user preference
-
Query Views – SQL UNION application
Dynamic formulas allow user specific aggregationsAggregate documents from multiple Dominodatabases into "single point of access"Returns federated data sets and documentsAllows for design of aggregated views
-
DB2
SQL UNION – Aggregated Views
SQL Query View
Notes Client
"Select A.Lastname, A.Firstname from Asia.Employee AUNIONSelect B.Lastname, B.Firstname from EMEA.Employee BUNIONSelect C.Lastname, C.Firstname from USA.Employee C"
NSFDB2 EMEA.nsf
DAVEmployee
NSFDB2 USA.nsf
DAVEmployee
NSFDB2 Asia.nsf
DAVEmployee
-
Scenario 5 – Aggregated Views
User tasks are dispersed over multiple workflowand project management applicationsUser is active in multiple teamroomsChallenge
user has to open many databases to get his job doneuser is not up to date if new tasks appear
Solutiona single view displays all documents a user needs from multiple databasesuse query view and SQL UNION to display federated (virtual) "documents"retrieving originating (real) documents accessible needs specific attention…
-
Demo 5
Design an aggregated view
Display external documents from multiple Domino databases in query view
Implement access mechanism to open external documents
-
Retrieve Documents From Federated Data - #1
View includes rows with federated data not related to documents of current database
which originate from (real) documents of external databases
Thus, document related Domino objects are not accessibledocument collection
document context
caret note ID not available
View entry objects (rows) originating from external documents doexist
Use caret category to identify row contextrequires unique value in sort column
limit of caret category is 99 characters
-
Include necessary data to access externaldocument to (hidden) designated column(s)
UNID / Note IDservernamepath or replica ID
Intercept QueryOpenDocument eventCreate backend document object that representsexternal documentOpen external document in Notes Client UI
Retrieve Documents From Federated Data - #2
-
Federated Data - Remarks
Dynamic resort has to be consideredwould change caret category
Wishlist for subsequent releasesidentifier for row selection context(other than caret category)provide data of external document automatically(servername, path or replica ID in addition to UNID)support response hierarchies
-
Enterprise Infrastructure Requirements
Prerequisites for aggregated viewsRequirements for applications
follow data structure policiesuse common set of application templates
Evaluate using DB2 as data store
-
Scenario 6 – Dynamic Column Data
Application has many fieldsUser needs are highly individualChallenge
many columns slow down view updatescustomize view functionality cannot be integrated in application designprivate views raise new issues
Solutionapplication lets the user specify data to displaya query view displays only the data specified by the user
-
Demo 6
Show field mapping
Show formula to alias DAV fields
-
Scenario 7 – Filter Categorized View
Application has views with complexcategory structuresUser needs complex and dynamicfilter mechanismsChallenge
full-text search breaks up category stucture
Solutionimplement user interface to set filtersa query view displays the filtered documents based on SQL selectcategory structure is retained
-
Demo 7
Show view update
Show profile document
Show formula to filter documents
-
Demo 7 Remarks
Make extensive use of filters to limit number of rows
customized query builderspersonalized viewsuser specific preferencessaved default values
-
Demo 7 Remarks
Consider multi value fieldsuse "like" operatoruse Notes multi value separator as separator for DAV
-
Scenario 8 – GCC Activity Manager
User needs complex relational filter mechanismsChallenge: portal like user experience
to collect relevant datato render relevant data in UI
Solutionimplement complex user interface to set filtersa query view displays the filtered documents based on SQL selectcategory structure is retained
-
Demo 8
Show GCC Activity ManagerExplain Architecture
-
Query Views - Performance
Performance lower than NSF for "as is" usageDB2 result sets
possible advantage for small result sets out of large document collectionsno view index storedlive SQL query on open or refresh
For certain scenariosquery views could help to handle performance issues –appropriate redesign has to be donequery views do not work locallylimit the number of records for result sets
-
Query View Remarks
Does NOT work locallyDB2 user mapping necessary for securityAlways use @DB2Schema to determine the schema name for queryDefault for maximum number of returned rows is 500 at server levelFederated data not available locallyFederated data must be available in DAV or DB2 tableQuery views with federated data do not update automatically
query is not re-executeduse F9 to update
-
Query View Summary
JOINS solve basic Notes problem with relations
New features offer the potential to create applications thatenable users to work more efficiently
Agregated views offer a new way to create a single point of access to all documents a user needs
Potential to address current problem scenarios with respect to performance/scalability
Keep the result set small
-
Agenda
1 Introduction2 DB2 Access Views (DAV)
scenariolive demo
3 Query Viewsscenariostechnology, tipsfederated data paradigmlive demos
4 Summary
-
Conclusion/Summary
Start thinking about scenarios in your companyStart consolidating your application landscapeNew integration options in Workplace contextLook into DB2 administrationGet skills in SQL programmingPrepare applications to utilize DAV and query viewsMake extensive use of filters
-
Related Lotusphere Activity
SessionsAD210 - Application Development and Deployment for DB2-Enabled IBM Lotus Domino ServerAD211 - Using IBM Lotus Domino 7 with DB2: A Real-World ExampleID105 - DB2 Setup and Administration for IBM Lotus Domino Server
BOF and Hands onHND202 - IBM Lotus Domino NSFDB2: Development and Integration Hands-OnMTG611 - Sharing Data Access View and Query View Development Experience
-
Q & A / Download
Questions and discussion welcome"Meet the Speaker" Swan Toucan 2 after the session"Gurupalooza"Sample code and updated slideshttp://gcc.upb.de/k-pool/LS2006_BP307
-
Contact Information
Dr. Ludwig W. Nastanskymailto: [email protected] Erdmannmailto: [email protected] of PaderbornGroupware Competence Centerhttp://gcc.upb.dePlease remember to fill out your evaluation!
BP307: IBM Lotus Domino 7 and DB2�Effective Use of Access and Query ViewsSpeakersAgenda1 IntroductionNotes/Domino & DB2 Integration, finally… ?Documents vs. Relational DataDocuments vs. Relational DataIntegration, Cooperation, FederationScenario 1 - Model Employee–Room RelationDomino challenge: Model Employee–Room RelationRemarkDemo 1: RelationsDB2 Access Views and Query ViewsAgendaDB2 Access ViewsDAV - OverviewDAV – Field DefinitionDAV – Architecture SummaryScenario 2: Integration #1Scenario 2: Integration #2Scenario 2: Mobile SalesforceDemo 2: DAV Design, Use & RemarksDAV UseDAV RemarksDAV SummaryAgendaQuery ViewsQuery Views - OverviewQuery Views – Data Federation #1 Query Views – Data Federation #2Query Views – Data Federation #3Query Views – SQL Query RulesQuery Views Are DynamicScenario 3 – Federated Data in ViewsSQL JOINDemo 3RemarksScenario 4 – Dynamic SelectionDemo 4Query Views – SQL UNION applicationSQL UNION – Aggregated ViewsScenario 5 – Aggregated ViewsDemo 5Retrieve Documents From Federated Data - #1Retrieve Documents From Federated Data - #2Federated Data - RemarksEnterprise Infrastructure RequirementsEnterprise Infrastructure RequirementsScenario 6 – Dynamic Column DataDemo 6Scenario 7 – Filter Categorized ViewDemo 7Demo 7 RemarksDemo 7 RemarksScenario 8 – GCC Activity ManagerDemo 8Query Views - PerformanceQuery View RemarksQuery View SummaryAgendaConclusion/SummaryRelated Lotusphere ActivityQ & A / DownloadContact InformationBackup SlidesSQL UNION RemarksDAV Design - #1DAV Design - #2DAV Design - #3DAV Design - #4Create new DAV for each formDAV DeploymentUser Mapping for SQL SecuritySequence of a SQL Query to DAVDAV JDBC AccessDAV JDBC AccessCreditsType section title hereAgendaTable of contentsLegal information