ChangeSummary / DAS RequirementsChangeSummary / DAS Requirements(SDO 3.0 Virtual F2F)(SDO 3.0 Virtual F2F)

Christophe Boutard (christophe.boutard@datadirect.com)François Huaulmé (francois.huaulme@datadirect.com)

Date: February 2009

AgendaAgenda

SDO + DAS current states

ChangeSummary scope

ChangeSummary operations clarification

New ChangeSummary format

2

Today SDO…Today SDO…

• … provides a ChangeSummary tracking modifications in the scope of a containment tree• Changes done on objects that are not in the data graph closure

are not taken into account• Containment relationships match XML oriented Data Models

• … considers creations/deletions/modifications in scope of the data graph (containment tree)

• … has the XML ChangeSummary format implying data graph XML serialization

• … will introduce the “Orphans” concept in v3.0:• Orphans allow dealing with non-closed data graphs (shared

instances, huge business oriented models…)• Orphans only exist in the XML (serialization artifacts)

3

The upcoming DAS…The upcoming DAS…

• … will provide data oriented services for reading SDOs and applying back changes to multiple backends (RDBMS, XML, services…)

• … defines an « applyChanges » operation relying on the SDO ChangeSummary content• DAS can be seen as the main consumer for ChangeSummary

• … will be able to deal with non-closed data graphs• Complex Business/Data models support• Relationships between multiple « read » services results• Non-closed data graphs changes committed in a single

transaction

4

AgendaAgenda

SDO + DAS current states

ChangeSummary scope

ChangeSummary operations clarification

New ChangeSummary format

5

ObjectiveObjective

Extend the XML ChangeSummary scope

• Put the « Orphan » objects changes in the scope of the XML ChangeSummary• Allow DASes to apply changes done on non-closed data graphs• Complete SDO 3.0 non-closed graphs support

• 50% already completed (« orphans » data objects XML serialization) • 50% TO BE DONE (« orphans » changes)

• Avoid breaking changes with SDO 2.1.x

6

How does it work? (1/2 )How does it work? (1/2 )

3 scenarios:• Orphans outside the scope of a ChangeSummary

continue to be serialized without changes as described in the first CD

• Orphans in the scope of a ChangeSummary having « logging=false » follow the rule above even if the main containment tree tracks changes

• Orphans in the scope of a ChangeSummary having « logging=true » are serialized and their changes included into the XML ChangeSummary• DAS « read » services return data graphs with a logging=true

ChangeSummary

7

How does it work? (2/2)How does it work? (2/2)

• In-memory orphans are just objects that are outside the containment tree to serialize

• At XML serialization time, orphan objects are added to orphan properties and then belong to the containment tree

• Orphans having changes are included in the XML ChangeSummary and can be referenced like any DataObject in SDO 2.1.x

8

XML ChangeSummary: Model exampleXML ChangeSummary: Model example

9

Company Employee

Address Project

Country

employees *

employeeOfTheMonth

project

address

country

1

1

1

1

containment

non-containment

XML ChangeSummary: SDO 2.1.x scopeXML ChangeSummary: SDO 2.1.x scope

10

Company Employee

Address Project

Country

employees *

employeeOfTheMonth

project

address

country

1

1

1

1

containment

non-containment

XML ChangeSummary: SDO 3.0 scopeXML ChangeSummary: SDO 3.0 scope

11

Company Employee

Address Project

Country

employees *

employeeOfTheMonth

project

address

country

1

1

1

1

containment

non-containment

addressOrphans

projectOrphans

XML ChangeSummary: proposed scopeXML ChangeSummary: proposed scope

12

Company Employee

Address Project

Country

employees *

employeeOfTheMonth

project

address

country

1

1

1

1

containment

non-containment

addressOrphans

projectOrphans

ExampleExample

DataObject megaCorp;megaCorp = das.readCompanyByName(“MegaCorp”);List employees = megaCorp.getList(“employees”);DataObject john =employees.get(0);DataObject mary = employees.get(1);DataObject jane = employees.get(2);

DataObject address = john.getDataObject(“address”); DataObject country;country = address.createDataObject(“country”);country.setString(“name”, “UK”);

DataObject project = mary.getDataObject(“project”);project.setString(“description”, “NEW description of PRJ002”);

13

Example: XML SchemaExample: XML Schema

<xsd:complexType name="CompanyType"> <xsd:sequence> <xsd:element name="employees” type="company:EmployeeType"

maxOccurs="unbounded"/> <xsd:element name="addressOrphans“ type="company:AddressType“

minOccurs="0“ maxOccurs="unbounded”  sdox:orphanHolder="true"/>

<xsd:element name="projectOrphans" type="company:ProjectType" minOccurs="0" maxOccurs="unbounded“

                          sdox:orphanHolder="true"/> </xsd:sequence> <xsd:attribute name="name" type="xsd:string"/> <xsd:attribute name="employeeOfTheMonth" type="xsd:string"

sdo:propertyType="company:EmployeeType"/></xsd:complexType>

14

Example: XML data graphExample: XML data graph

<company:company name="MegaCorp" employeeOfTheMonth="#/sdo:datagraph/company:company[1]/employees[2]">

<employees name="John Jones" SN="E0001" address="#/sdo:datagraph/company:company[1]/addressOrphans[1]“project="#/sdo:datagraph/company:company[1]/projectOrphans[1]"/>

<employees name="Mary Smith" SN="E0002" address="#/sdo:datagraph/company:company[1]/addressOrphans[1]“

              project="#/sdo:datagraph/company:company[1]/projectOrphans[2]"/> <employees name="Jane Doe" SN="E0003"

address="#/sdo:datagraph/company:company[1]/addressOrphans[2]“              project="#/sdo:datagraph/company:company[1]/projectOrphans[2]"/>

<addressOrphans number="123" street="main street"> <country name="UK"/> </addressOrphans> <addressOrphans number="456" street="16th street"> <country name="US"/> </addressOrphans>

<projectOrphans code="PRJ001" description="description of PRJ001"/> <projectOrphans code="PRJ002" description="NEW description of PRJ002"/></company:company>

15

Example: XML ChangeSummaryExample: XML ChangeSummary

<changeSummary delete="" create="#/sdo:datagraph/company:company[1]/addressOrphans[1]/country[1]">

<company:company sdo:ref="#/sdo:datagraph/company:company[1]" employeeOfTheMonth="#/sdo:datagraph/company:company[1]/employees[1]">

<addressOrphans number="123" street="main street" sdo:unset="country"/> <addressOrphans number="456" street="16th street">

    <country name="US"/> </addressOrphans>

<projectOrphans code="PRJ001" description="description of PRJ001"/> <projectOrphans code="PRJ002" description="description of PRJ002"/>

</company:company>

</changeSummary>

16

AgendaAgenda

SDO + DAS current states

ChangeSummary scope

ChangeSummary operations clarification

New ChangeSummary format

17

ObjectivesObjectives

• Make events recorded in the ChangeSummary more easily usable by a DAS

• Objects lifecycle should be independent of the data graph• « Creations » and « Deletions » should describe the state of

objects with a bigger scope than a data graph

• Distinguish « moving » an object from « deleting » it

18

Extending the DataObject lifecycleExtending the DataObject lifecycle

SDO 3.0 and DAS tend to consider DataObjects outside data graphs

• The SDO 3.0 Keys lead to extend the DataObjects lifecycle• DataObjects can

• be identified independently from a data graph• appear in multiple data graphs

• In SDO + DAS architecture objects uniqueness must be ensured at the backend level• isCreated in a data graph does not always mean isCreated at

the backend level

19

Creations (1/2)Creations (1/2)

• In DAS architectures DataObjects can be instantiated by:• User (client side): insert new instances into the backend• DAS (server side): “read” operations results

• Today “ChangeSummary.isCreated()” means “is added to a data graph”• This does not allow differencing objects instantiated by users

(really created) from objects returned by a DAS

• SDO must not solve DAS issues but should provide mechanisms that make things possible…

20

Creations (2/2)Creations (2/2)

• Containment relationships would continue following the SDO 2.1.x rules• Adding an object on a containment relationships would be

tracked as a creation without considering previous actions done on the object (e.g. DataObject.delete())

• Non containment relationships to orphan objects need a new rule• An object added to a non containment relationship will not be

considered as a creation if it was not in the scope of a ChangeSummary at the time ChangeSummary.beginLogging() was called

21

Delete vs. MoveDelete vs. Move

• Today SDO does not allow distinguishing “moves” from “deletes”• From a DAS (backend) point of view

• “Delete” must produce a delete order (e.g. SQL delete for RDBMS)• “Move” must produce a reference modification

• Explicit deletions (DataObject.delete()) must always be tracked as delete orders without considering the containment information

• The needed clarification only affects “moves”…

22

How to track “Moves”?How to track “Moves”?

• Containment relationships• Moving a contained DataObject should continue to be tracked

as a deletion in the context of the old container• Then setting the moved object to another relationship would

follow the rules defined for “creations”

• Non containment relationships to “orphan” objects• A “nullification” must be tracked for the old owner of the

relationship• The moved object is not marked as deleted

23

AgendaAgenda

SDO + DAS current states

ChangeSummary scope

ChangeSummary operations clarification

New ChangeSummary format

24

Objectives – QuestionsObjectives – Questions

Objectives:

• Provide a more efficient XML ChangeSummary format

• Provide a format that can be more easily used by a DAS• SDO Keys mapped to a backend identity• Optimistic Concurrency Control (OCC) based on old values + Keys

Questions:

• Should SDO 3 allow a ChangeSummary to be sent without anything but only the changes?• Avoid data graph XML serialization (less verbose).• Currently XML ChangeSummary only stores old values. New values are

serialized in the data graph.

• A more efficient XML ChangeSummary format could leverage SDO 3 Keys • Objects identity can move from technical state to a business representation• Use of identity in sdo:ref?

25

How to represent the object identity?How to represent the object identity?

Use SDO 3 identity in the ChangeSummary. But how?

• A canonical representation of the key?• General format:

• {type_name, idProperty0=idProperty0Value, …, idPropertyN=idPropertyNValue}

• Examples• sdo:ref=”{abc:Type, id=2, id2=3, objectId={abc:ObjectType, objId=10}}”• sdo:ref=”{abc:Type, id=2, id2=3}”

• A new reprensentation of the “sdo:ref”?• XML elements containing Keys

• Something else?

26

Improve SDO ChangeSummary Improve SDO ChangeSummary informationinformation

• SDO 2.1: ChangeSummary is the “copy” of a data graph when beginLogging() was called.

• Should ChangeSummary be…• An action log? A list of actions (new, move, delete, set…)

• Can highly reduce the old values size when dealing with many=true properties.

• An optimized action log? • A log that prunes unnecessary recorded actions.

27

QUESTIONS ?QUESTIONS ?

28