Chapter 10
Database Systems with XML Support
Oracle12c
DB2 11.0
Microsoft SQL Server 2016
PostgreSQL 9.6
11-2Lecture "XML and Databases" - Dr. Can Türker
Example:XML Schema Document
11-3Lecture "XML and Databases" - Dr. Can Türker
Example: XML Documents
Gunter SaakeIngo SchmittCan TürkerObjektdatenbanken
Konzepte, Sprachen, Architekturen
International Thomson Publishing
Dieses Lehrbuch ...
Meike KlettkeHolger MeyerXML & Datenbanken
XML-Dokumente effizient speichernund verarbeiten
dpunktMit der wachsenden ...
XML Document book1.xml XML Document book2.xml
Can TürkerSQL:1999 & SQL:2003
Objektrelationales SQL, SQLJ & SQL/XML
dpunktSQL ist ...
XML Document book3.xml
Gunter SaakeKai-Uwe SattlerDatenbanken & JavaJDBC, SQLJ, ODMG & JDOdpunkt2 Das Zusammenspiel ...
XML Document book4.xml
11-4Lecture "XML and Databases" - Dr. Can Türker
Oracle: Architecture (1)
Figure taken from Oracle® XML Developer's KitProgrammer's Guide 11g Release 1 (11.1) April 2008
11-5Lecture "XML and Databases" - Dr. Can Türker
Oracle: Architecture (2)
Figure taken from Oracle® XML DBDeveloper’s Guide 11g Release 1 (11.1) October 2007
11-6Lecture "XML and Databases" - Dr. Can Türker
Mapping XML to Databases
Approaches
– XML column approach: column of type XMLTYPE
– XML table approach: table of type XMLTYPE
Exploit object-relational extensions
– XMLTYPE as predefined object type with SQL/XML functions as methods
– Intermedia-Text package with full-text functionality
– DBMS_XMLDOM package with DOM methods
– DBMS_XMLSCHEMA package with methods for creating and managing XML schemas
– DBMS_XMLGEN package with methods to generate XML from SQL
Storage options
– Text-based (unstructured as CLOB)
– Binary (compact in binary format)
– Schema-based (object-relational)
– Hybrid (semi-structured)
Storage Options (1)
11-7Lecture "XML and Databases" - Dr. Can Türker
Figure taken from Oracle® XML DBDeveloper’s Guide 11g Release 1 (11.1) October 2007
Storage Options (2)
11-8Lecture "XML and Databases" - Dr. Can Türker
Figure taken from Oracle® XML DBDeveloper’s Guide 11g Release 1 (11.1) October 2007
11-9Lecture "XML and Databases" - Dr. Can Türker
XML Column vs. XML Table
CREATE TABLE ( XMLTYPE)[XMLTYPE [COLUMN] [STORE AS { OBJECT RELATIONAL | CLOB () | BINARY XML ()}][XMLSCHEMA ELEMENT [#]]]
CREATE TABLE OF XMLTYPE [XMLTYPE STORE AS { OBJECT RELATIONAL | CLOB () | BINARY XML() }][XMLSCHEMA ELEMENT [#]]]
schema-based storage text-based storage
Default: BINARY XML
Table with XML column
Table with XML row
binary storage
11-10Lecture "XML and Databases" - Dr. Can Türker
User-Defined Function to Read XML Documents
CREATE DIRECTORY xmldir AS 'c:\xmldir';GRANT READ ON DIRECTORY xmldir TO PUBLIC WITH GRANT OPTION;
CREATE FUNCTION getDocument(filename VARCHAR2) RETURN CLOBAUTHID CURRENT_USER IS
xbfile BFILE;xclob CLOB;
BEGINxbfile := BFILENAME('xmldir', filename);DBMS_LOB.open(xbfile);DBMS_LOB.createTemporary(xclob, TRUE, DBMS_LOB.session);DBMS_LOB.loadFromFile(xclob, xbfile, DBMS_LOB.getLength(xbfile));DBMS_LOB.close(xbfile);
RETURN xclob;END;/
11-11Lecture "XML and Databases" - Dr. Can Türker
XML Column
Table with XML column
Insert XML value as XML column value
CREATE TABLE book (content XMLTYPE
) ;
INSERT INTO bookVALUES (XMLTYPE('
Meike KlettkeHolger Meyer ...
'));
INSERT INTO book VALUES (XMLTYPE(getDocument('book1.xml')));
11-12Lecture "XML and Databases" - Dr. Can Türker
XML Table
Table with XML rows
Insert XML value as XML row value
CREATE TABLE book OF XMLTYPE;
INSERT INTO bookVALUES (XMLTYPE('
Meike KlettkeHolger Meyer ...
'));
INSERT INTO book VALUES (XMLTYPE(getDocument('book1.xml')));
11-13Lecture "XML and Databases" - Dr. Can Türker
XML Schema Handling
Package DBMS_XMLSCHEMA provides methods to register, compile, generate, and remove XML schemas
DBMS_XMLSCHEMA.registerSchema('schema-url', 'xml-schema');
DBMS_XMLSCHEMA.generateSchema('schema', 'type-name');
DBMS_XMLSCHEMA.deleteSchema('schema-url', deleteoption);
DELETE_RESTRICT CONSTANT NUMBER := 1;DELETE_INVALIDATE CONSTANT NUMBER := 2;DELETE_CASCADE CONSTANT NUMBER := 3;DELETE_CASCADE_FORCE CONSTANT NUMBER := 4;
DBMS_XMLSCHEMA.compileSchema('schema-url');
11-14Lecture "XML and Databases" - Dr. Can Türker
Schema-based XML Table
Register XML schema
Create XML table based on this schema
Alternative syntax
BEGINDBMS_XMLSCHEMA.registerSchema('book.xsd', getDocument('book.xsd'));
END;/
CREATE TABLE book OF XMLTYPEXMLSCHEMA "book.xsd" ELEMENT "book";
CREATE TABLE book OF XMLTYPEELEMENT "book.xsd#book";
11-15Lecture "XML and Databases" - Dr. Can Türker
Queries
Support for SQL/XML functions
– XMLQUERY, XMLTABLE, XMLEXISTS, XMLCAST
– XMLELEMENT, XMLFOREST, XMLCONCAT, XMLAGG
– XMLPI, XMLCOMMENT, XMLSERIALIZE, XMLPARSE
Plus additional functions
– XMLROOT
– XMLCOLATTVAL
– XMLCDATA
Full-text search
11-16Lecture "XML and Databases" - Dr. Can Türker
XMLTYPE
creates XML value
Example:
XMLTYPE( )
Joe
Jim
2000
3500
name salaryemployee SELECT XMLTYPE(' ' || 12 * salary || ' ') AS worker
FROM employee;
24000
42000
WORKER
creates XML value from an XQuery expression
Example:
7-17Lecture "XML and Databases" - Dr. Can Türker
Datenbanken & JavaGunter SaakeKai-Uwe Sattler
BOOK
ObjektdatenbankenGunter Saake Ingo SchmittCan Türker
XMLQUERY
XMLQUERY( [PASSING [BY VALUE] ]RETURNING CONTENT [NULL ON EMPTY])
:= AS ""
SELECT XMLQUERY('{$i//title/text()}{for $a in $i//author return {$a/text()}}'
PASSING VALUE(b) AS "i" RETURNING CONTENT) AS bookFROM book bWHERE XMLQUERY('$i/book[author[1] = "Gunter Saake"]' PASSING VALUE(b) AS "i"
RETURNING CONTENT) IS NOT NULL;
7-18Lecture "XML and Databases" - Dr. Can Türker
XMLTABLE
creates SQL table from an XQuery expression. Columns option allows user-defined mapping to the table columns. Without this option, an XML table is created.
Example:
XMLTABLE ([] [PASSING [BY VALUE] ] [COLUMNS ])
:= [PATH ] [DEFAULT]
3-8266-0258-7
3-89864-219-4
3-89864-148-1
ISBN
3-89864-228-3
Objektdatenbanken
SQL:1999 & SQL:2003
XML & Datenbanken
BOOKTITLE
Datenbanken & Java
SELECT t.*FROM book b, XMLTABLE('$i' PASSING VALUE(b) AS "i" COLUMNS
isbn VARCHAR(30) PATH '/book/@isbn', booktitle VARCHAR(50) PATH '/book/title') AS t;
returns true if the XQuery expression yields a non-null value
Example:
XMLEXISTS( [PASSING [BY VALUE] ])
:= AS ""
7-19Lecture "XML and Databases" - Dr. Can Türker
XMLEXISTS
Objektdatenbanken
Datenbanken & Java
BOOKTITLE
SELECT XMLQuery('$i//title/text()' PASSING VALUE(b) AS "i"RETURNING CONTENT) AS booktitle
FROM book bWHERE XMLEXISTS('$i//author[text() = "Gunter Saake"]' PASSING VALUE(b) AS "i");
11-20Lecture "XML and Databases" - Dr. Can Türker
XMLCAST
Target data type can be any (alpha)numeric SQL data-type, including CLOB/BLOB plus REF XMLTYPE
Example:
XMLCAST( AS )
Objektdatenbanken
SQL:1999 & SQL:2003
XML & Datenbanken
BOOKTITLE
Datenbanken & Java
SELECT XMLCAST(XMLQUERY('$i//book/title' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS CLOB) AS booktitle
FROM book b;
11-21Lecture "XML and Databases" - Dr. Can Türker
XMLELEMENT
created XML element
Example:
XMLELEMENT(NAME [, XMLATTRIBUTES()][, ])
:= [AS ]
Joe
Jim
WORKER
Joe
Jim
2000
3500
name salaryemployee SELECT XMLELEMENT(NAME "worker", XMLATTRIBUTES(salary), name) AS worker
FROM employee;
Joe24000
Jim42000
WORKER
11-22Lecture "XML and Databases" - Dr. Can Türker
XMLFOREST
creates forest of XML elements with optional attributes
Example:
XMLFOREST()
:= [AS ]
Joe
Jim
2000
3500
name salaryemployee SELECT XMLFOREST(name, XMLELEMENT(NAME "euro", 12*salary) AS income) AS worker
FROM employee;
11-23Lecture "XML and Databases" - Dr. Can Türker
XMLCONCAT
concatenates XML elements to a forest
Example:
XMLCONCAT()
Joe
Jim
2000
3500
name salaryemployee
Joe24000
Jim42000
WORKER
SELECT XMLCONCAT(XMLELEMENT(NAME "employee", name), XMLELEMENT(NAME "euro", 12*salary))
AS workerFROM employee;
11-24Lecture "XML and Databases" - Dr. Can Türker
XMLAGG
aggregates XML elements
Example:
XMLAGG([ORDER BY ] )
SELECT name, XMLAGG(XMLELEMENT(NAME "salary", salary))AS income
FROM employeeGROUP BY name;
Joe
Jim
2000
3500
name salaryemployee
Jim 5000
2000
35005000
Joe
Jim
NAME INCOME
7-25Lecture "XML and Databases" - Dr. Can Türker
XMLPARSE, XMLSERIALIZE, XMLCOMMENT & XMLPI
creates XML value (WELLFORMED avoids validity checking)
converts XML value to an SQL string or LOB (in case of schema-based XML values, SHOW DEFAULTS uses the defaults of the given XML schema)
creates XML comment node
creates XML processing instruction node
XMLCOMMENT ()
XMLPI (NAME [, ]))
XMLPARSE({DOCUMENT | CONTENT} [WELLFORMED])
XMLSERIALIZE({DOCUMENT | CONTENT} [AS ] [ENCODING ] [VERSION ] [{SHOW|HIDE} DEFAULTS])
7-26Lecture "XML and Databases" - Dr. Can Türker
XMLCDATA, XMLROOT & XMLCOLATTVAL
creates XML node of form ]]>
creates XML node (if available the XML attributes version and standalone are set)
creates XML elements of the form Value
XMLCDATA()
XMLROOT(, VERSION {|NO VALUE} [, STANDALONE {YES|NO [VALUE]}])
XMLCOLATTVAL()
:= [AS ]
Index Types
– Functional (value)
index
– Full-text index
– XML index
available for binary or text-based XML columns/tables and creates a set of secondary indexes
Path index on all XML tags and fragments
Value index on the order of the document nodes
Value index on the values of the document nodes
Command to avoid using XML index in query
Command to avoid using XML index in query and performing XQuery optimization
CREATE INDEX xmlfunctionalidx ON book b (XMLCAST(XMLQUERY( '$i//@year' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS INT));
11-27Lecture "XML and Databases" - Dr. Can Türker
Indexing (1)
CREATE INDEX xmlfulltextidx ON book b (VALUE(b)) INDEXTYPE IS CTXSYS.CONTEXT;
CREATE INDEX xmlidx ON book b (VALUE(b)) INDEXTYPE IS XDB.XMLIndex;
SELECT /*+ NO_XMLINDEX_REWRITE */ ...
SELECT /*+ NO_XML_QUERY_REWRITE */ ...
11-28Lecture "XML and Databases" - Dr. Can Türker
Indexing (2)
Query exploiting the functional index
Query exploiting the full-text index
Query exploiting the XML index
SELECT XMLQUERY('' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS book
FROM book bWHERE XMLCAST(XMLQUERY('$i//@year' PASSING VALUE(b) AS "i" RETURNING CONTENT)
AS INT) > 2000;
SELECT XMLQUERY('$i//book/title' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS booktitleFROM book bWHERE XMLEXISTS('$i//book/title' PASSING VALUE(b) AS "i");
SELECT XMLQUERY('$i//@isbn' PASSING VALUE(b) AS "i" RETURNING CONTENT) AS isbnFROM book bWHERE CONTAINS(VALUE(b), 'Das book', 0) > 0ORDER BY SCORE(0) DESC;
Manipulation
Persistent updates only possible via SQL UPDATE
Text-based storage supports only complete replacement of XML values ; other storage types also support partial replacement
SQL functions use XPath expressions for selective update
– updateXML – replace XML node
– insertChildXML – insert XML node as child of a given XML element node
– insertChildXMLbefore – insert XML elements as child of a given element node before the child elements of the given type
– insertChildXMLafter – insert XML elements as child of a given element node after the child elements of the given type
– insertXMLbefore – insert XML node before the given node
– insertXMLafter – insert XML node after the given node
– appendChildXML – insert XML node as last child element node of the given node
– deleteXML – remove XML node
XQuery Update Facility is not supported!
1-29Lecture "XML and Databases" - Dr. Can Türker
11-30Lecture "XML and Databases" - Dr. Can Türker
Manipulation — UPDATEXML
updates the part of the XML value given by the XPath query
Example:
UPDATEXML(, [, ])
:= ,
...dpunkt ...
... International Thomson Publishing ...
...dpunkt ...
book SYS_NC_ROWINFO$
...dpunkt ...
UPDATE book bSET VALUE(b) = UPDATEXML(VALUE(b), '//publisher[text()="dpunkt"]/@city', 'Zürich');
11-31Lecture "XML and Databases" - Dr. Can Türker
Manipulation — DELETEXML
deletes a part of the XML value given by the XPath query
Example:
DELETEXML(, [,])
UPDATE book bSET VALUE(b) = DELETEXML(VALUE(b),
'//book[@isbn="3-89864-148-1"]/author[text()="Holger Meyer"]');
...
Meike KlettkeXML & Datenbanken ...
book SYS_NC_ROWINFO$
11-32Lecture "XML and Databases" - Dr. Can Türker
XML Views
Based on the concept of object views using XMLTYPE as object type
Example: CREATE VIEW dpunktbook OF XMLTYPE WITH OBJECT ID DEFAULT AS
SELECT VALUE(b)
FROM book b
WHERE XMLEXISTS('$i//publisher[text()="dpunkt"]' PASSING VALUE(b) AS "i");
11-33Lecture "XML and Databases" - Dr. Can Türker
Export SQL Data to XML
Standard SQL-XML mapping using
– Table content mapped to elements
– Column values of each row are mapped to child elements of a element
– Structured (complex) columns are mapped to elements with corresponding child (hierarchically nested) elements
– Collections mapped to list of elements
– Object reference and foreign keys mapped to ID/IDREFs
User-defined transformation from SQL to XML possible via XSLT
… … …
DBMS_XMLGEN.getXML('query')
11-34Lecture "XML and Databases" - Dr. Can Türker
Summary: Oracle
XML Storage
Model extensional, object-relational
Schema validation possible
Storage Type binary, text-based, schema-based
SQL-XML Mapping Using SQL/XML functions, schema generators, XML views
XML Data Type available
XML Indexing
Value/Functional Index available
Full-text Index available
Path Index available (implicitly as part of XML indexes)
Queries and Manipulation
Queries SQL/XML with XQuery support
Full-text Search using Intermedia-Text package
Manipulation complete replacement using SQL/XML functions in SQL UPDATE (selective SQL UPDATE possible in case of schema-based XML values but XQuery Update Facility not supported)
11-35Lecture "XML and Databases" - Dr. Can Türker
IBM DB2
Database
File-system
File-system
File-system
FileSystem
DB2
XML DocumentsApplication
11-36Lecture "XML and Databases" - Dr. Can Türker
Mapping XML to Databases
After IBM has deprecated the DB2 XML Extender, DB2 supports only an XML column approach
Table with column of type XML
– Binary/text-based storage
– XML schema validation possible
11-37Lecture "XML and Databases" - Dr. Can Türker
Table with XML Column
Register XML schema
Create table with XML column
Load XML documents from a file
REGISTER XMLSCHEMA 'book.xsd' FROM 'file://C:\XMLDIR\book.xsd' AS book.xsd COMPLETE
IMPORT FROM 'C:\XMLDIR\book.del' OF DEL XML FROM C:\XMLDIRXMLVALIDATE USING XDS DEFAULT book.xsdINSERT INTO book
CREATE TABLE book (Id INT PRIMARY KEY,content XML
)
1, 2, 3, 4,
File book.del
11-38Lecture "XML and Databases" - Dr. Can Türker
Queries
Use XQuery in SQL via SQL/XML functions
– XMLQUERY, XMLTABLE, XMLEXISTS, XMLCAST
– XMLELEMENT, XMLFOREST, XMLCONCAT, XMLAGG
– XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI
– XMLTEXT, XMLDOCUMENT, XMLVALIDATE, XSLTRANSFORM
– XMLROW, XMLGROUP
Use SQL in XQuery
– XQUERY db2-fn:xmlcolumn ('t1.xml1')
yields value of column xml1 of table t1 as a node sequence provided the column is of type XML
– XQUERY db2-fn:sqlquery ('SELECT xml1 FROM t1')
yields value of the column xml1 of table t1 as a node sequence provided the column is of type XML
Full-text search requires installation and activation of DB2 Text Extender
creates XML value from an XQuery expression. Difference to Oracle: resulting XML value is never NULL; an empty sequence is delivered as such!
Example:
7-39Lecture "XML and Databases" - Dr. Can Türker
Datenbanken & JavaGunter SaakeKai-Uwe Sattler
BOOK
ObjektdatenbankenGunter Saake Ingo SchmittCan Türker
XMLQUERY
XMLQUERY( [PASSING [BY REF] ][RETURNING SEQUENCE [BY REF] [EMPTY ON EMPTY])
:= AS ""
SELECT XMLQUERY('{$i//title/text()}{for $a in $i//author return {$a/text()}}'
PASSING content AS "i") AS bookFROM book bWHERE XMLEXISTS ('$i/book[author[1] = "Gunter Saake"]' PASSING content AS "i");
7-40Lecture "XML and Databases" - Dr. Can Türker
XMLTABLE
creates SQL table from an XQuery expression. Without the COLUMNS option, a table with an XML column is created. Difference to Oracle: XMLQUERY und XMLTABLE can receive the XML input via reference.
Example:
XMLTABLE ([] [PASSING [BY REF] < xquery-argument-list>] [COLUMNS ])
:= [BY REF] [PATH ] [DEFAULT] }
3-8266-0258-7
3-89864-219-4
3-89864-148-1
ISBN
3-89864-228-3
Objektdatenbanken
SQL:1999 & SQL:2003
XML & Datenbanken
BOOKTITLE
Datenbanken & Java
SELECT t.*FROM book, XMLTABLE('$i' PASSING content AS "i" COLUMNS
isbn VARCHAR(30) PATH '/book/@isbn', booktitle VARCHAR(50) PATH '/book/title') AS t;
return true if the XQuery expression yields a non-null value
Example:
XMLEXISTS( [PASSING [BY REF] ])
:= AS ""
7-41Lecture "XML and Databases" - Dr. Can Türker
XMLEXISTS
Objektdatenbanken
Datenbanken & Java
BOOKTITLE
SELECT XMLQuery('$i//title/text()' PASSING content AS "i") AS booktitleFROM bookWHERE XMLEXISTS('$i//author[text() = "Gunter Saake"]' PASSING content AS "i");
11-42Lecture "XML and Databases" - Dr. Can Türker
XMLCAST
Difference to Oracle: SQL data type XML can be used here as target type!
Example:
XMLCAST( AS )
Objektdatenbanken
SQL:1999 & SQL:2003
XML & Datenbanken
BOOKTITLE
Datenbanken & Java
SELECT XMLCAST(XMLQUERY('$i//book/title' PASSING content AS "i") AS CLOB) AS booktitleFROM book;
11-43Lecture "XML and Databases" - Dr. Can Türker
XMLELEMENT
creates XML-Element (XMLBINARY determines the encoding of the input)
Example:
Joe
Jim
WORKER
Joe
Jim
2000
3500
name salaryemployee SELECT XMLELEMENT(NAME "worker", XMLATTRIBUTES(salary), name) AS worker
FROM employee;
XMLELEMENT(NAME [, XMLATTRIBUTES()][, ] [XML-content-option-list>])
:= [AS ]
:= {NULL | EMPTY} ON NULL | XMLBINARY [USING] {BASE64 | HEX}
11-44Lecture "XML and Databases" - Dr. Can Türker
XMLFOREST
creates forest of XML elements
Example:
XMLFOREST( [XML-content-option-list>])
:= [AS ]
Joe
Jim
2000
3500
name salaryemployee SELECT XMLFOREST(name, XMLELEMENT(NAME "euro", 12*salary) AS income) AS worker
FROM employee;
Joe24000
Jim42000
WORKER
11-45Lecture "XML and Databases" - Dr. Can Türker
XMLCONCAT
concatenates XML elements to a forest
Example:
XMLCONCAT()
Joe
Jim
2000
3500
name salaryemployee
Joe24000
Jim42000
WORKER
SELECT XMLCONCAT(XMLELEMENT(NAME "employee", name), XMLELEMENT(NAME "euro", 12*salary))
AS workerFROM employee;
11-46Lecture "XML and Databases" - Dr. Can Türker
XMLAGG
aggregates XML elements
Example:
XMLAGG([ORDER BY ] )
SELECT name, XMLAGG(XMLELEMENT(NAME "salary", salary))AS income
FROM employeeGROUP BY name;
Joe
Jim
2000
3500
name salaryemployee
Jim 5000
2000
35005000
Joe
Jim
NAME INCOME
7-47Lecture "XML and Databases" - Dr. Can Türker
XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI
creates XML value. Default option STRIP WHITESPACE eliminates spaces from text nodes which consist only out of spaces
converts XML value to an SQL string. Option INCLUDING XMLDECLARATION additionally inserts the XML declaration . Default: EXCLUDING.
creates XML comment node
creates XML processing instruction node
XMLCOMMENT ()
XMLPI (NAME [, ]))
XMLPARSE (DOCUMENT [{STRIP|PRESERVE} WHITESPACE])
XMLSERIALIZE([CONTENT] [AS ] [VERSION ˈ1.0ˈ] [{EXCLUDING|INLUDING} XMLDECLARATION])
7-48Lecture "XML and Databases" - Dr. Can Türker
XMLTEXT, XMLDOCUMENT, XMLVALIDATE, XSLTRANSFORM
creates XML text node
creates XML document node
returns copy of XML value enriched by information from the schema validation, including the default values
transforms XML value using an XSL style sheet
XMLTEXT()
XMLDOCUMENT(
7-49Lecture "XML and Databases" - Dr. Can Türker
XMLROW
returns XML document node where each value expression is attached as a child node. Option AS ATTRIBUTES creates attribute nodes instead of child nodes. Option ROW can be used to rename the document element.
XMLROW( [OPTION ]
:= [AS ""]
:= ROW "" | AS ATTRIBUTES
SELECT name, XMLROW(name, salary) AS incomeFROM employee;
Joe
Jim
2000
3500
name salaryemployee
Joe2000
Jim3500
Joe
Jim
NAME INCOME
7-50Lecture "XML and Databases" - Dr. Can Türker
XMLGROUP
returns XML document element node with child element nodes for each expression. Option AS ATTRIBUTES creates attribute nodes instead of element nodes. Option ROWSET can be used to rename the document element node.
XMLROW( [ORDER BY ] [OPTION ]
:= [AS ""]
:= ROOT "" | ROW "" | AS ATTRIBUTES
SELECT name, XMLGROUP(salary) AS incomeFROM employeeGROUP BY name;
Joe
Jim
2000
3500
name salaryemployee
Jim 5000
2000
35005000
Joe
Jim
NAME INCOME
11-51Lecture "XML and Databases" - Dr. Can Türker
XQuery returns single-column table with a row for each item of the resulting sequence
XMLQuery returns table with a (possibly empty) sequence for each table row
XQuery vs. XMLQuery
Gunter SaakeKai-Uwe Sattler
Meike Klettke
Holger Meyer
Can Türker
Gunter Saake
Kai-Uwe Sattler
Gunter SaakeKai-Uwe Sattler
Meike KlettkeHolger Meyer
Can Türker
Gunter SaakeKai-Uwe Sattler
XQUERY db2-fn:sqlquery('SELECT content FROM book')//book[@year > 2000]/author;
SELECT XMLQUERY('$i//book[@year > 2000]/author' PASSING content as "i") FROM book;
Index Types
– XML index
– Full-text index
Query using the XML index
Query using (and requiring) the full-text index
CREATE INDEX xmlbooktitleidx ON book (content) GENERATE KEY USING XMLPATTERN '/book/title' AS SQL VARCHAR(50);
11-52Lecture "XML and Databases" - Dr. Can Türker
Indexing
CREATE INDEX xmlfulltextidx FOR TEXT ON book(content);
SELECT XMLQUERY('$i/book/title' PASSING content AS "i") AS booktitleFROM bookWHERE XMLEXISTS('$i/book/title' PASSING content AS "i");
SELECT id, INTEGER(SCORE(content, '(XML OR book)')*1000) AS score FROM book WHERE CONTAINS(content, '(XML OR book)')=1ORDER BY score DESC;
11-53
Only complete replacement of XML values possible via SQL UPDATE
XQuery Update Facility supported to manipulate a copy of an XML value
Manipulation
Jim Bob Meike KlettkeHolger MeyerXML & Datenbanken
XML-Dokumente effizient speichernund verarbeiten
dpunktMit der wachsenden ...
UPDATE book SET content = XMLQUERY('transform copy $pi := $i modify do
insert document { Jim Bob } before $pi//book/author[1] return $pi' PASSING content AS "i")
WHERE XMLEXISTS('$i/book[author/text() = "Holger Meyer"]' PASSING content AS "i");
11-54Lecture "XML and Databases" - Dr. Can Türker
Summary: IBM DB2
XML Storage
Model extensible, object-relational
Schema Validation possible
Storage Type binary/text-based (XML stored as LOBs)
SQL-XML Mapping XQUERY + XMLQUERY
XML Data Type available
Indexing
Value Index available (as part of the XML index)
Full-text Index using Text Extender*
Path Index available (as part of the XML index)
Queries and Manipulation
Queries SQL/XML with XQuery support
Full-text Search using Text Extender*
Manipulation complete replacement via SQL UPDATE but no selective update; XQuery Update Facility not supported
*Text Extender must explicitly be installed and activated!
11-55Lecture "XML and Databases" - Dr. Can Türker
Microsoft SQL Server - Architecture
Database
MS SQL Server
SQLOLEDB
XML DocumentsApplication
Internet Information Server (IIS)
ADOMiddleware
11-56Lecture "XML and Databases" - Dr. Can Türker
Mapping XML to Databases
Storage Types
– Native (binary) storage
– Text-based storage as CLOB
– Model-based storage using the EDGE approach (extended SQL)
– Schema-based storage using STORED queries (extended SQL)
Data type XML with XQuery methods
– Query() – evaluates XQuery and returns a value of type XML
– Value() – evaluates XQuery and returns a scalar SQL value
– Exist() – returns true if the XQuery yields a non-empty result
– Modify() – updates a value of type XML
Integrated usage of SQL and XQuery
– Access to SQL data in XQuery using the functions sql:column("columnname") and sql:variable("variablename"), respectively
– Evaluation of XQuery in SQL using XML methods above
11-57Lecture "XML and Databases" - Dr. Can Türker
Native Storage – Table Definition
Register XML schema
Create table with XML column
Insert XML document from file
CREATE XML SCHEMA COLLECTION bookXSD AS '…'
INSERT INTO bookSELECT 1, xColFROM (SELECT *
FROM OPENROWSET (BULK 'C:\XMLDIR\book1.xml', SINGLE_BLOB) AS xCol) AS R(xCol)
CREATE TABLE book (Id INT PRIMARY KEY,content XML (bookXSD)
)
11-58Lecture "XML and Databases" - Dr. Can Türker
Native Storage - Indexing
Create primary XML index
– creates clustered index with entries of the form (ID, ORDPATH, TAG, NODETYPE, VALUE, PATH_ID, …)
– needed to created secondary XML indexes
Secondary XML index types:
– Path index (Path, Value)
– Property index (Primary key , Path, Value)
– Value index (Value, Path)
Secondary XML (path plus property) indexes
Full-text index
CREATE PRIMARY XML INDEX contentidx ON book (content)
CREATE XML INDEX contentpathidx ON book (content) USING XML INDEX contentidx FOR PATH CREATE XML INDEX contentpropertyidx ON book (content) USING XML INDEX contentidx FOR PROPERTY
PATH | PROPERTY | VALUE
CREATE UNIQUE INDEX bookkeyidx ON book (Id)CREATE FULLTEXT CATALOG xmldbftcat AS DEFAULTCREATE FULLTEXT INDEX ON book (content) KEY INDEX bookkeyidx
11-59Lecture "XML and Databases" - Dr. Can Türker
Example: XQuery using SQL data to create new XML values
Native Storage – Queries (1)
SELECT content.query('{//title}{for $a in //author return {$a}}') AS book
FROM bookWHERE content.exist('/book[author[1] = "Gunter Saake"]') = 1
ObjektdatenbankenGunter SaakeIngo SchmittCan Türker
Datenbanken & JavaGunter SaakeKai-Uwe Sattler
book
11-60Lecture "XML and Databases" - Dr. Can Türker
Example: Query using the XML index
Example: Query using the full-text index
Native Storage – Queries (2)
SELECT content.query('//author') AS authorsFROM bookWHERE content.exist('/book[author[1] = "Gunter Saake"]') = 1
Gunter SaakeKai-Uwe Sattler
Gunter SaakeIngo SchmittCan Türker
authors
SELECT id, contentFROM book WHERE CONTAINS(content, '(XML OR book)')
11-61Lecture "XML and Databases" - Dr. Can Türker
Example: insert new element
Example: update attribute value
Native Storage – Manipulation
UPDATE bookSET content.modify('replace value of (//publisher[. = "dpunkt"]/@city)[1] with "Zürich"')
UPDATE bookSET content.modify('insert Unknown before (//author)[1]')
11-62Lecture "XML and Databases" - Dr. Can Türker
Model-based Storage using EDGE (1)
SQL extension OPENXML transforms XML values to SQL tables
OPENXML without the WITH clause returns an EDGE table (result see next foil)
DECLARE @xmldoctext VARCHAR(8000) = ' Meike KlettkeHolger MeyerXML & DatenbankenXML-Dokumente effizient speichern und verarbeitendpunkt ...'
DECLARE @hdoc INT
EXEC sp_xml_preparedocument @hdoc OUTPUT, @xmldoctext
SELECT * FROM OPENXML(@hdoc, '', 0)
EXEC sp_xml_removedocument @hdoc
11-63Lecture "XML and Databases" - Dr. Can Türker
Model-based Storage using EDGE (2)
BIGINT BIGINT INT NVARCHAR NVARCHAR NVARCHAR NVARCHAR BIGINT NTEXT
id parentid nodetype localname prefix namespaceuri datatype prev text
0 NULL 1 book NULL NULL NULL 1 NULL1 NULL 7 xml NULL NULL NULL NULL NULL2 1 2 version NULL NULL NULL NULL NULL3 1 2 encoding NULL NULL NULL NULL NULL5 0 2 isbn NULL NULL NULL NULL NULL6 0 2 year NULL NULL NULL NULL NULL7 0 1 author NULL NULL NULL NULL NULL8 0 1 author NULL NULL NULL 7 NULL9 0 1 title NULL NULL NULL 8 NULL10 0 1 subtitle NULL NULL NULL 9 NULL11 0 1 publisher NULL NULL NULL 10 NULL12 11 2 city NULL NULL NULL NULL NULL13 11 3 #text NULL NULL NULL NULL dpunkt14 0 1 description NULL NULL NULL 11 NULL15 2 3 #text NULL NULL NULL NULL 116 3 3 #text NULL NULL NULL NULL ISO-8859-117 5 3 #text NULL NULL NULL NULL 3-89864-148-118 6 3 #text NULL NULL NULL NULL 200319 7 3 #text NULL NULL NULL NULL Meike Klettke20 8 3 #text NULL NULL NULL NULL Holger Meyer21 9 3 #text NULL NULL NULL NULL XML & Datenbanken22 10 3 #text NULL NULL NULL NULL XML-Dokumente ...23 12 3 #text NULL NULL NULL NULL Heidelberg24 14 3 #text NULL NULL NULL NULL ...
11-64Lecture "XML and Databases" - Dr. Can Türker
Schema-based Storage using STORED Queries (1)
OPENXML supports custom mappings
OPENXML with the WITH clause creates user-defined table (result see next foil)
DECLARE @xmldoctext VARCHAR(8000) = ' Meike KlettkeHolger MeyerXML & DatenbankenXML-Dokumente effizient speichern und verarbeitendpunkt ...'
DECLARE @hdoc INT
EXEC sp_xml_preparedocument @hdoc OUTPUT, @xmldoctext
SELECT * FROM OpenXML(@hdoc, '//book', 0) WITH (title NVARCHAR(3000) './title',isbn NVARCHAR(15) './@isbn',publisher NVARCHAR(200) './publisher')
SELECT * FROM OpenXML(@hdoc, '//author', 0) WITH (name NVARCHAR(200) '.',isbn NVARCHAR(200) '../@isbn')
EXEC sp_xml_removedocument @hdoc
11-65Lecture "XML and Databases" - Dr. Can Türker
Schema-based Storage using STORED Queries (2)
Resulting table "book"
Resulting table "author"
title isbn publisher
XML & Datenbanken 3-89864-148-1 dpunkt
name isbn
Meike Klettke 3-89864-148-1
Holger Meyer 3-89864-148-1
11-66Lecture "XML and Databases" - Dr. Can Türker
Mapping Databases to XML
Variant 1: Standard mapping using SQL SELECT and FOR XML
– FOR XML AUTO: Generates XML element names and reflects foreign keys in hierarchies
– FOR XML RAW: Maps to ROW-XML elements und XML attributes
– FOR XML PATH: Generates complex XML elements using XPath
– FOR XML EXPLICIT: User controls XML mapping (EDGE)
Variant 2: User-defined XML view
– Use XML schema
– Annotate the schema with information over tables and columns
– Access XML view via IIS functionality or ADO (ActiveX Data Objects)
11-67Lecture "XML and Databases" - Dr. Can Türker
FOR XML AUTO
SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML AUTO
XML_XXX: NTEXT (CLOB)
SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML AUTO, ELEMENTS
Meike KlettkeXML & Datenbanken dpunkt3-89864-148-1Holger MeyerXML & Datenbanken dpunkt3-89864-148-1
XML_XXX: NTEXT (CLOB)
Option XMLSCHEMA ('uri') generates XML Schema
11-68Lecture "XML and Databases" - Dr. Can Türker
FOR XML RAW
SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML ROW
XML_XXX: NTEXT (CLOB)
SELECT name, title, publisher, book.isbnFROM author JOIN book ON (author.isbn = book.isbn)FOR XML AUTO, ELEMENTS
Meike KlettkeXML & Datenbanken dpunkt3-89864-148-1Holger MeyerXML & Datenbankendpunkt3-89864-148-1
XML_XXX: NTEXT (CLOB)
11-69Lecture "XML and Databases" - Dr. Can Türker
FOR XML PATH
SELECT book.isbn AS "@isbn", name AS "author", publisher AS "title/@publisher", title AS "title/text()"FROM author JOIN book ON (author.isbn = book.isbn)FOR XML PATH
Meike KlettkeXML & DatenbankenHolger MeyerXML & Datenbanken
XML_XXXX: NTEXT (CLOB)
SELECT book.isbn AS "@isbn", name AS "author", publisher AS "title/@publisher", title AS "title/text()"FROM author JOIN book ON (author.isbn = book.isbn)FOR XML PATH ('books'), ROOT ('bookstore')
Meike KlettkeXML & DatenbankenHolger MeyerXML & Datenbanken
XML_XXXX: NTEXT (CLOB)
11-70Lecture "XML and Databases" - Dr. Can Türker
User-Defined XML Views
Annotated XML schema
– Namespace: xmlns:sql="urn:schemas-microsoft-com:mapping-schema"
– Annotation sql:relation: Relationship between XML and a table
– Annotation sql:field: Relationship between XML and a table column
– Annotation sql:relationship: Foreign key relationship
XML view access via XPath
– Some XPath construct are not supported
– XPath transformed to corresponding SQL
11-71Lecture "XML and Databases" - Dr. Can Türker
User-Defined XML Views (Example)
Book Publisher Title isbn
Thomson Objektdatenbanken 3-8266-0258-7
Objektdatenbanken 3-8266-00258-7 Thomson
Definition ofXML structure
Definition of table columns
11-72Lecture "XML and Databases" - Dr. Can Türker
Manipulation using Updategrams
Schema-based mapping required
– Updates formulated in an XML document
– New namespace: xmlns:updg="urn:schemas-microsoft-com:xml-updategram" Element before: Definition of the state to be updated
Element after: Definition of the new state
– Update operations Insert: empty element
Delete: empty element
Update: both elements are non-empty
Example:
Thomson
International Thomson Publishing
11-73Lecture "XML and Databases" - Dr. Can Türker
Summary: SQL Server
XML Storage
Model relational
Schema inline DTD or XML schema
Storage Type native: XML column text-based: CLOB columnmodel-based: OPENXMLuser-defined, schema-based: OPENXML
SQL-XML Mapping automatic: FOR XMLuser-defined: XSD annotations
XML Data Type available
Indexing
Value Index available
Full-text Index available
Path Index available
Queries and Manipulation
Queries SQL extensions (query und value are not SQL/XML compatible), XQuery
Full-text Search supported
Manipulation XML method modify or using updategrams; XQuery Update Facility not supported
11-74Lecture "XML and Databases" - Dr. Can Türker
PostgreSQL - Architecture
Database
PostgreSQL Server
XML DocumentsApplication
11-75Lecture "XML and Databases" - Dr. Can Türker
Storage and Queries
XML data type
– Text-based storage
– No schema validation
SQL/XML functions
– XMLEXISTS, XMLELEMENT, XMLFOREST, XMLCONCAT, XMLAGG
– XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI
– Plus XMLROOT
– No support for XMLQUERY und XMTABLE
– XPath instead XQUERY support
Full-text search
11-76Lecture "XML and Databases" - Dr. Can Türker
XML
creates XML value
Example:
XML( )
Joe
Jim
2000
3500
name salaryemployee
SELECT XML('' || 12 * salary || '') AS worker
FROM employee;
24000
42000
worker
11-77Lecture "XML and Databases" - Dr. Can Türker
XPATH (1)
extracts part(s) of an XML value using an XPath expression. Resulting XML sequence represented by a value of the SQL type SET OF.
Example:
XPATH(, )
{"Gunter Saake"}
{"Can Türker"}
{"Meike Klettke"}
firstauthor
{"Gunter Saake"}
Gunter Saake
Can Türker
Meike Klettke
firstauthorunnested
Meike Klettke
SELECT XPATH('//author[1]', content) AS firstauthor, TEXT(UNNEST(XPATH('//author[1]', content))) AS firstauthorunnested
FROM book;
11-78Lecture "XML and Databases" - Dr. Can Türker
XPATH (2)
{"Meike Klettke""Holger Meyer"
}
authors
{"Gunter Saake""Ingo Schmitt""Can Türker"}
{"Can Türker"}
{"Gunter Saake ""Kai-Uwe Sattler"}
SELECT XML('' || TEXT(XPATH('//author', content))|| '') AS authorsFROM book b;
11-79Lecture "XML and Databases" - Dr. Can Türker
XPATH (3)
Meike KlettkeHolger Meyer
authors
Gunter SaakeIngo SchmittCan Türker
Can Türker
Gunter SaakeKai-Uwe Sattler
SELECT XMLELEMENT(NAME "authors", XPATH('//author', content)) AS authorsFROM book b;
returns true if the XQuery expression yields a non-null value
Example:
XMLEXISTS( PASSING [BY REF] )
7-80Lecture "XML and Databases" - Dr. Can Türker
XMLEXISTS
{Objektdatenbanken}
{Datenbanken & Java}
booktitle
SELECT XPATH('//title/text()', content) AS booktitleFROM bookWHERE XMLEXISTS('//author[text() = "Gunter Saake"]' PASSING content);
11-81Lecture "XML and Databases" - Dr. Can Türker
XMLELEMENT
creates XML element (XMLBINARY defines encoding)
Example:
Joe
Jim
worker
Joe
Jim
2000
3500
name salaryemployee SELECT XMLELEMENT(NAME "worker", XMLATTRIBUTES(salary), name) AS worker
FROM employee;
XMLELEMENT(NAME [, XMLATTRIBUTES()][, ])
:= [AS ]
11-82Lecture "XML and Databases" - Dr. Can Türker
XMLFOREST
creates forest of XML elements
Example:
XMLFOREST()
:= [AS ]
Joe
Jim
2000
3500
name salaryemployee SELECT XMLFOREST(name, XMLELEMENT(NAME "euro", 12*salary) AS income) AS worker
FROM employee;
Joe24000
Jim42000
worker
11-83Lecture "XML and Databases" - Dr. Can Türker
XMLCONCAT
concatenates XML elements to a forest
Example:
XMLCONCAT()
Joe
Jim
2000
3500
name salaryemployee
Joe24000
Jim42000
worker
SELECT XMLCONCAT(XMLELEMENT(NAME "employee", name), XMLELEMENT(NAME "euro", 12*salary))
AS workerFROM employee;
11-84Lecture "XML and Databases" - Dr. Can Türker
XMLAGG
aggregates XML elements
Example:
XMLAGG([ORDER BY ] )
SELECT name, XMLAGG(XMLELEMENT(NAME "salary", salary))AS income
FROM employeeGROUP BY name;
Joe
Jim
2000
3500
name salaryemployee
Jim 5000
2000
35005000
Joe
Jim
name income
7-85Lecture "XML and Databases" - Dr. Can Türker
XMLPARSE, XMLSERIALIZE, XMLCOMMENT, XMLPI
creates XML value
converts XML value to SQL string
creates XML comment node
creates XML processing instruction node
XMLCOMMENT ()
XMLPI (NAME [, ]))
XMLPARSE ({DOCUMENT|CONTENT} )
XMLSERIALIZE({DOCUMENT|CONTENT} AS )
XMLROOT(, VERSION {|NO VALUE} [, STANDALONE {YES|NO [VALUE]}])
7-86Lecture "XML and Databases" - Dr. Can Türker
XMLROOT
creates XML node (if available XML attribute version and standalone are set too)
11-87Lecture "XML and Databases" - Dr. Can Türker
Indexing
Index support
– Functional Index
(Value index)
– Full-text index
Query using the functional index
Query using full-text index
CREATE INDEX xmlfunctionalidx ON book USING BTREE (((XPATH('//author/text()', content))[1]::TEXT));
SELECT * FROM bookWHERE (XPATH('//author/text()', content))[1]::TEXT = 'Gunter Saake';
CREATE INDEX xmlfulltextidx ON book USING GIN(to_tsvector('german', content::TEXT));
SELECT *FROM bookWHERE to_tsvector(content::TEXT) @@ plainto_tsquery('german','Gunter');
11-88Lecture "XML and Databases" - Dr. Can Türker
Export Database to XML
Standard mapping of SQL to XML using the following functions
Variant (xxx for table, query, schema, database):
– xxx_to_xmlschema returns corresponding XML schema
– xxx_to_xml_and_xmlschema returns XML value and corresponding XML schema
table_to_xml(tablename text, nulls boolean, tableforest boolean, targetns text)
query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)
schema_to_xml(schemaname text, nulls boolean, tableforest boolean, targetns text)
database_to_xml(nulls boolean, tableforest boolean, targetns text)
11-89Lecture "XML and Databases" - Dr. Can Türker
TABLE_TO_XML (1)
exports table to XML
1) null values are mapped to empty XML elements or are ignored
2) document: false maps row to XML sequence; true to an XML document element
3) optional namespace
Example:
TABLE_TO_XML(, , , )
Joe2000.00
Jim3500.00
table_to_xml
SELECT TABLE_TO_XML('employee', true, true, '');
11-90Lecture "XML and Databases" - Dr. Can Türker
TABLE_TO_XML (2)
Example:
Joe2000.00
Jim3500.00
table_to_xml
SELECT TABLE_TO_XML('employee', true, false, '');
11-91Lecture "XML and Databases" - Dr. Can Türker
TABLE_TO_XMLSCHEMA
table_to_xmlschema
SELECT TABLE_TO_XMLSCHEMA('employee', true, false, '');
11-92Lecture "XML and Databases" - Dr. Can Türker
QUERY_TO_XML
Example:
Joe24000.00
Jim42000.00
query_to_xml
SELECT QUERY_TO_XML('SELECT name, 12*salary AS annualsalary FROM employee', true, false, '')
11-93Lecture "XML and Databases" - Dr. Can Türker
Summary: PostgreSQL
XML Storage
Model extensible, object-relational
Schema not supported
Storage Type text-based
SQL-XML Mapping using SQL/XML functions
XML Data Type available
Indexing
Value Index available
Full-text Index using tsearch package
Path Index not supported
Queries and Manipulation
Queries SQL/XML without XMLQUERY und XMLTABLE but with XPATH support
Full-text Search using tsearch package
Manipulation complete replacement using SQL UPDATE but no selective update; XQuery Update Facility not supported
4-94Lecture "XML and Databases" - Dr. Can Türker
Comparison – SQL/XML FunctionsTyp SQL:2003 Oracle DB2 MSSQL PostgreSQL
XML ✓ ✓ (XMLTYPE) ✓ ✓ ✓
XMLPARSE ✓ ✓ ✓ — (CAST) ✓
XMLSERIALIZE ✓ ✓ ✓ — (CAST) ✓
XMLCOMMENT ✓ ✓ ✓ — ✓
XMLPI ✓ ✓ ✓ — ✓
XMLDOCUMENT ✓ — ✓ — —
XMLTEXT ✓ — ✓ — —
XMLVALIDATE ✓ — ✓ — —
XMLQUERY ✓ ✓ ✓ — (query) — (XPATH)
XMLTABLE ✓ ✓ ✓ — —
XMLEXISTS ✓ ✓ ✓ — (exist) ✓ (XPATH)
XMLCAST ✓ ✓ ✓ — (CAST) —
XMLELEMENT ✓ ✓ ✓ — ✓
XMLFOREST ✓ ✓ ✓ — ✓
XMLCONCAT ✓ ✓ ✓ — ✓
XMLAGG ✓ ✓ ✓ — ✓
XMLROOT — ✓ — — ✓
XMLCDATA — ✓ — — —
XMLCOLATTVAL — ✓ — — —
XMLROW — — ✓ — —
XMLGROUP — — ✓ — —
FORXML — — — ✓ —
OPENXML — — — ✓ —
Top Related