XQuery 1.0 Formal Semantics

Presented by:

Michael Ryabtzev

Kravtsov Valentin

• The goal of the formal semantics is to complement the specification , by defining the meaning of expressions with mathematical rigor.- Clarifies the intended meaning- Ensures that no corner cases are left out- Provides a reference for implementation

Introduction to the Formal Semantics

XQuery Processing Model

– The static semantics describes how the output type is inferred from the core operator tree and the input type.

– The dynamic semantics describes how the output instance produced by using the accessors and constructors defined in the XQuery Data Model.

– The normalization rules transform full XQuery into small core language, which is easier to define, implement and optimize.

Formal Semantics defines static semantics, dynamic semantics and normalization rules:

Formal notation: Expr Value

This is read, “Evaluation of expression Expr yields value Value”We call this an evaluation judgment.

Definition of values:Definition of expressions:

Expr ::= Value

| Expr < Expr

| Expr + Expr

| if(Expr) then Expr

else Expr

Value ::== Boolean | Integer

Boolean ::== fn:true() | fn:false()

Integer ::== 0 | 1 | -1 | 2 | …

Dynamic Semantics:

First 5 rules of Evaluations:

_______________

Value Value

Expr0 Integer0

Expr1 Integer1_________

Expr0 < Expr1 Integer0 < Integer1

(VALUE)

(LT)

Expr0 Integer0

Expr1 Integer1________

Expr0 + Expr1 Integer0 + Integer1(SUM)

Conclusion

Hypotheses

Dynamic Semantics:

Evaluations:

Expr0 fn:true()

Expr1 Value_________

If (Expr0) then Expr1 else Expr2 Value (IF-TRUE)

Expr0 fn:false()

Expr2 Value_________

If (Expr0) then Expr1 else Expr2 Value(IF-FALSE)

Dynamic Semantics:

Example of proof tree for expression:

Note that Expr2 = 5 + 6 did not appear in the hypotheses, which formalizes the intuition that the else branch is not evaluated when the condition is true.

(VALUE)

1 1 (VALUE)

2 2 (LT)

1 < 2 fn:true()

(VALUE)

3 3 (VALUE)

4 4 (SUM)

3 + 4 7

(IF-

TRUE)if (1 < 2) then 3 + 4 else 5 + 6 7

Dynamic Semantics:

Environment – a judgment relating expression to a value.

Formal notation: dynEnv |- Expr ValueThis is read, “In environment dynEnv, the evaluation of expression Expr yields the value Value”.

ØThe initial environment with an empty map

dynEnv.varValue(Var1 Value1 , … , Varn Valuen)

The environment that maps Vari to Valuei

dynEnv + varValue(Var Value)

The environment identical to dynEnv except that Var maps to Value

dynEnv.varValue(Var)The value of Var in dynEnv

dom(dynEnv.varValue)The set of variables mapped in dynEnv

Notations that manipulate environments:

Environments:

Examples of environments:

dynEnv0 = Ø

dynEnv1 = dynEnv0 + varValue(x 1) = varValue(x 1)

dynEnv2 = dynEnv1 + varValue(y 2) = varValue(x 1, y 2)

dynEnv3 = dynEnv2 + varValue(x 3) = varValue(x 3, y 2)

Note that binding a variable in the environment overrides any previous binding for the same variable.

dynEnv2.varValue(x) = 1

dom(dynEnv0.varValue) = Ø

dom(dynEnv3.varValue) = { x , y }

Environments:

We can now formalize variables and let expressions:

Expr ::= Value

| Expr < Expr

| Expr + Expr

| if(Expr) then Expr else Expr

| $Var

| let $Var := Expr return Expr

Environments:

The 5 rules we gave before need to be revised to mention the environment. Example for revision of 2 first rules:

dynEnv |- Env Expr0 Integer0

dynEnv |- Expr1 Integer1_________

dynEnv |- Expr0 < Expr1 Integer0 < Integer1

_______________dynEnv |-Value Value

Environments:

We also need to add two new rules:

dynEnv.varValue(Var) = Value______

dynEnv |- $Var Value

dynEnv |- Expr0 Value0

dynEnv + varValue(Var Value0) |- Expr1 Value1

dynEnv |- let $Var := Expr0 return Expr1 Value1

Note that evaluation of Expr1 is preformed after assigning Value0 to Var.

Environments:

Expr ::= Value

| Expr < Expr

| Expr + Expr

| if(Expr) then Expr else Expr

| $Var

| let $Var := Expr return Expr

| let $Var as Type := Expr return Expr

For now we use only two types:Type ::= xs:boolean | xs:integer

Lets extend our grammar to represent relationship between values and types:

Matching Values and Types:

Matching judgment:

Value matches Type

This is read, “Value Value matches the type Type”.

_____________________Integer matches xs:integer

The inference rules for literal expressions:

(INT-MATCH)

(BOOL-MATCH)______________________Boolean matches xs:boolean

Matching Values and Types:

dynEnv |- Expr0 Value0

Value0 matches Type

dynEnv + varValue(Var Value0) |- Expr1 Value1

dynEnv |- let $Var as Type := Expr0 return Expr1 Value1

The first and the third hypotheses are the same as those in the rule for let without type declarations. The second hypothesis asserts that Value0 matches the declared type.

Matching Values and Types:

Error judgment:

dynEnv |- Expr raises Error

This is read, “In the environment dynEnv the evaluation of expression Expr raises error Error”.

We classify errors as either type errors or dynamic errors.

Error ::= typeErr | dynErr

Errors:

Expr ::= Value

| Expr < Expr

| Expr + Expr

| if(Expr) then Expr else Expr

| $Var

| let $Var := Expr return Expr

| let $Var as Type := Expr return Expr

| Expr idiv Expr

Lets extend our grammar to illustrate the semantics of errors:

Errors:

dynEnv |- Expr0 Value0

dynEnv |- Expr1 Value1

Value1 0____________

dynEnv |-Expr0 idiv Expr1 Value0 idiv Value1

(IDIV)

The evaluation rule for division is similar to that for addition:

dynEnv |- Expr1 0____________

dynEnv |-Expr0 idiv Expr1 raises dynErr

We now add rules that indicate when errors should be raised:

(IDIV-ERR)

Note that the rule does not require evaluation of the dividend in Expr0 to discover such an error.

Errors:

dynEnv |- Expr0 Value0

not (Value0 matches xs:integer) ____

dynEnv |-Expr0 < Expr1 raises typeErr

(LT-LEFT-TYPE-ERR)

The following rules show how type errors may be raised during evaluation of arithmetic and comparison:

dynEnv |- Expr1 Value1

not (Value1 matches xs:integer) ____

dynEnv |-Expr0 < Expr1 raises typeErr

(LT-RIGHT-TYPE-ERR)

The rules for sum, division operators and if expression are omitted but similar.

Errors:

dynEnv |- Expr0 Value0

not (Value0 matches Type) ____

dynEnv |-let $Var as Type := Expr0 return Expr1 raises typeErr

(LEFT-TYPE-ERR)

The let expression with a type declaration raises an error if the value is not of the declared type:

Errors:

dynEnv |- Expr0 raises Error_____

dynEnv |-Expr0 < Expr1 raises Error

The following rules illustrate how an error is propagated from the sub-expression in which it occurs to the containing expression:

dynEnv |- Expr1 raises Error_____

dynEnv |-Expr0 < Expr1 raises Error

Errors:

dynEnv |- Expr0 raises Error_____

dynEnv |-let $Var as Type := Expr0 return Expr1 raises Error

In let expression, an error is propagated if it arises in the first or second expression:

dynEnv |- Expr0 Value0

dynEnv + varValue(Var Value0) |- Expr1 raises Error___

dynEnv |- let $Var as Type := Expr0 return Expr1 raises ErrorNote that we bind the variable to it’s value before checking whether second expression raises an error.

Errors:

Here is example of evaluation that raises an error:

_______________________________(VALUE)

dynEnv0 |- 0 0 (VAR)

dynEnv1 |- $x 0 (IDIV-ERR)

dynEnv1 |- 1 idiv 0 raises dynErr (SUM-RIGHT-ERR)

dynEnv1 |- $x + (1 idiv $x) raises dynErr (LET)

dynEnv0 |- let $x := 0 return $x + (1 idiv $x) raises dynErrwhere dynEnv0 = Ø, dynEnv1 = varValue(x 0).

Errors:

Formal notation: statEnv |- Expr Type

This is read, “In environment statEnv, expression Expr has type Type”

We call this a typing judgment.

ØThe initial static environment with an empty map

statEnv + varType(Var : Type)

The environment identical to statEnv except that Var maps to Type

statEnv.varType(Var)The type of Var in statEnv

dom(statEnv.varType)The set of variables mapped in statEnv

Static Semantics:

Value ::= Boolean | IntegerExpr ::= Value | Expr < Expr | Expr + Expr | if(Expr) then Expr else Expr | $Var | let $Var := Expr return Expr | let $Var as Type := Expr return Expr | Expr idiv ExprType ::= xs:boolean | xs:integer

Here is the XQuery grammar that we have built so far:

Static Semantics:

Examples of static rules:

statEnv |- Expr0 :xs:integer statEnv |- Expr1 :xs:integer___

statEnv |- Expr0 < Expr1 :xs:boolean

(LT-STATIC)

statEnv |- Expr0 :xs:booleanstatEnv |- Expr1 : Type

statEnv |- Expr2 : Type________statEnv |- if (Expr0) then Expr1 else Expr2:Type

(IF-STATIC)

statEnv |- Expr0 : Type0

statEnv + varType(Var:Type0) |- Expr1 : Type1___

statEnv |- let $Var as Type0 := Expr0 return Expr1:Type1

(LET-DECL-STATIC)

Static Semantics:

Recall the two dynamic rules for let:

Note that an important consequence of static analysis is that we do not need to check for type errors at evaluation time matches judgment in the first rule and the entire second rule are redundant when static typing is in effect.

dynEnv |-Expr0 Value0

Value0 matches TypedynEnv + varValue(Var Value0) |-Expr1 Value1

dynEnv |- let $Var as Type := Expr0 return Expr1 Value1

dynEnv |- Expr0 Value0

not (Value0 matches Type) ____dynEnv |- let $Var as Type := Expr0 return Expr1 raises typeErr

Static Semantics:

dynEnv matches statEnv

If dom(dynEnv.varValue) = dom(statEnv.varType)

And for every x in that domain,

dynEnv.varValue(x) matches statEnv.varType(x)

Lets define the relationship the dynamic environment and corresponding static environment

Type Soundness:

Theorem: Type soundness for valuesif

dynEnv matches statEnvdynEnv |- Expr ValuestatEnv |- Expr : Type

thenValue matches Type

Theorem: Type Soundness for Errorsif

dynEnv matches statEnvdynEnv |- Expr raises ErrorstatEnv |- Expr : Type

thenError typeErr

Properties of static type checkingType Soundness:

Formal notation: [FullExpr]Expr == Expr

The Expr subscript indicates that full XQuery expression can be normalized by the rule.

For example:

[let $Var as Type := Expr0 where Expr1 return Expr2]Expr

==

let $Var as Type := [Expr0]Expr return if ([Expr1]Expr) then [Expr2] else ()

Normalization:

We start with the formal notation for values:

Value ::= () | Item (,Item)*Item ::= AtomicValue | NodeValue

Values and Typing:

AtomicValue ::= xs:integer(String) | xs:boolean(String) | xs:string(String) | xs:date(String)

NodeValue ::= element ElementName TypeAnnotation? {Value} | text {String}ElementName ::= QNameTypeAnnotation ::= of type TypeNameTypeName ::= QName

Values and Typing:

<user> <name><first>Mary</first><last>Doe</last><name> <rating>A</rating></user>

element user of type User { element name of type Name { element first of type xs:string { “Mary” } , element last of type xs:string { “Doe” } } , element rating of type xs:string { “A” }}

Values and Typing:

The formal notation for types:

At the top level one can define elements and types

Definition ::= define element ElementName TypeAnnotation | define type TypeName TypeDerivation

TypeAnnotation ::= of type TypeName

TypeDerivation ::= restrics AtomicTypeName | restrics TypeName {Type} | {Type}

Type ::= none() | empty() | ItemType | Type , Type | Type | Type | Type Occurrence

Occurrence ::= ? | + | *

SimpleType ::= AtomicType | SimpleType | SimpleType | SimpleType Occurrence

Values and Typing:

ItemType ::= NodeType | AtomicType

NodeType ::= ElementType | text()

AtomicType ::= AtomicTypeName

AtomicTypeName ::= xs:string | xs:integer | xs:date | xs:boolean

ElementType ::= element((ElementName (,TypeName)?)?)

Values and Typing:

Example:<xs:element name=“article" type=“Article"/> <xs:complexType name=“Article"> <xs:sequence> <xs:element name=“name" type=“xs:string"/> <xs:element name=“reserve_price" type=“PriceList" minOccurs="0“ maxOccurs=“unbounded”/> </xs:sequence> </xs:complexType> <xs:simpleType name=“PriceList”> <xs:list itemType = “xs:decimal” /></xs:simpleType>

define element article of type Articledefine type Article{ element (name, xs:string), element (reserve_price,PriceList) * }define type PriceList restricts xs:anySimpleType {xs:decimal * }

Values and Typing:

• Now lets define matching to relate complex XML values with complex types.

Example:<reserve_price>10.00 20.00 30.00</reserve_price>

Before validation, this element is represented by the following untyped XML value:element reserve_price { text { “ 10.00 20.00 30.00 “ } }

After validation, this element is represented by the following typed XML value:element reserve_price of type PriceList { 10.0 20.0 30.0 }

element reserve_price of type PriceList {10.0 20.00 30.00} matches element(reserve_price)

Matching and Subtyping:

Rule for matching elements against element types:

statEnv |- ElementType yields element (ElementName1, TypeName1)

statEnv |- ElementName substitutes for ElementName1

statEnv |- TypeName derives from TypeName1

statEnv |- Value matches TypeName_____________

statEnv |- element ElementName of type TypeName {Value} matches

ElementType

Matching and Subtyping:

Yields judgment:ElementType yields element (ElementName,ElementType)

statEnv |- define element ElementName of type ElementType

statEnv |- element(ElementName) yields

element(ElementName,ElementType)

____________________________________________________

statEnv |- element(ElementName,ElementType) yields

element(ElementName,ElementType)

Example: element(article) yields element(article,Article)

Example: element(reserve_price,PriceList) yields element(reserve_price,PriceList)

Matching and Subtyping:

____________________________________________________

statEnv |-

element(*,ElementType) yields element(*,ElementType)

____________________________________________________

statEnv |-

element() yields element(*,xs:anyType)

Example: element(*,PriceList) yields element(*,PriceList)

Example: element() yields element(*,xs:anyType)

Matching and Subtyping:

Substitution judgment:

ElementName1 substitutes for ElementName2

____________________________________________________

statEnv |-

ElementName substitutes for ElementName

____________________________________________________

statEnv |-

ElementName substitutes for *

Matching and Subtyping:

Derives judgment:TypeName1 derives from TypeName2

define type TypeName restricts TypeName1

TypeName derives from TypeName1

define type TypeName restricts TypeName1 {Type}

TypeName derives from TypeName1

___________________________________________

TypeName derives from TypeName

TypeName1 derives from TypeName2

TypeName2 derives from TypeName3

TypeName1 derives from TypeName3

Matching and Subtyping:

Matches judgment:Value matches Type

__________________________________() matches empty()

AtomicTypeName derives from xs:integer

Integer matches AtomicTypeName

Value1 matches Type1

Value2 matches Type2

Value1, Value2 match Type1, Type2

Matching and Subtyping:

Value matches Type1

Value matches Type1 | Type2

Value matches Type2

Value matches Type1 | Type2

Value matches empty() | TypeValue matches Type?

Value matches Type , Type*Value matches Type+

Value matches Type+ ?Value matches Type*

Matching and Subtyping:

Example: element (article)+ subtype element (article)*element (*,NewUser) subtype element (*,User)

Subtyping judgment:Type1 subtype Type2

holds if every value that matches the first type also matches the second.

Subtyping is defined by logical equivalence:Type1 subtype Type2

if and only ifValue matches Type1 implies Value matches Type2

Matching and Subtyping:

Example: (element(), text())* , element() , text() (element(), text())+

element(), text() , (element() , text())*

FLWOR Expressions – the “workhorse” of XQuery.

Example of XQuery FLWOR Expression:

for $i in $I, $j in $Jlet $k := $i + $jwhere $k >= 5return ( $i , $j )

FLWOR Expressions:

Grammar for the FLWOR Expressions

Expr := … previous expressions… | FLWRExprFLWRExpr := Clause + return ExprClause := ForExpr | LetExpr | WhereExprForExpr := for ForBinding (,ForBinding)*LetExpr := let LetBinding (,LetBinding)*WhereExpr := where ExprForBinding := $Var TypeDeclaration? PositionalVar? in ExprLetBinding := $Var TypeDeclaration? := ExprTypeDeclaration := as SequenceTypePositionalVar := at $VarSequenceType := ItemType Occurence

FLWOR Expressions:

Normalization – it is easier to define the static and dynamic semantics of an expressions if it is small and doesn’t do too much.

for $i in $I, $j in $Jlet $k := $i + $jwhere $k >= 5return ( $i , $j )

for $i in $I return for $j in $J return let $k := $i + $j return

if ($k >= 5) then return ( $i , $j )else ()

Normalization

FLWOR Expressions - Normalization:

[for ForBinding1 ... ForBindingn return Expr]Expr

==[for ForBinding1 ... [ForBindingn return Expr]Expr]Expr’ (n>1)

FLWOR Expressions - Normalization:

[Clause1 ... Clausen return Expr]Expr

== [Clause1 return ... [Clausen return Expr]Expr]Expr’ (n>1)

[let LetBinding1 ... LetBindingn return Expr]Expr

==[let LetBinding1 ... [LetBindingn return Expr]Expr]Expr’ (n>1)

let $k:=5 for $i in (1,2,3)let $k := $k + $i return ($k)

FLWOR Example:

<result> 6 7 8 </result>

Results in …

FLWOR Expressions:

dynEnv |- Expr0 (Item1, ... , Itemn)

dynEnv + varValue(Var Item1) |- Expr1 Value1

… dynEnv + varValue(Var Item1) |- Expr1 Value1____

dynEnv |- for $Var in Expr0 return Expr1 (Value1,… ,Valuen)

Dynamic semantics

Note that each evaluation is independent of every other evaluation. When Var is mapped to Item1 has no effect on the evaluation when Var is mapping to Itemn.

FLWOR Expressions:

statEnv |- Expr0 ItemType*

statEnv + varType(Var:ItemType) |- Expr1 : Type____

statEnv |- for $Var in Expr0 return Expr1 : Type*

Static semantics

FLWOR Expressions:

Note that here is a static typing rule assumes the type of the input sequence is already factored.

((xs:integer, xs:string) | xs:integer)*subtype (xs:integer | xs:string)*

and(element(title), element(author)+)

subtype (element(title) | element(author))+

Example of types and their factorization:

FLWOR Expressions:

Prime ::= ItemType | none() | Prime | Prime

FLWOR Expressions:

Quantifier ::= 1 exactly one Type 1 = Type | ? zero or one Type ? = Type ? | + one or more Type + = Type + | * zero or more Type * = Type *

prime(ItemType) = ItemTypeprime(empty()) = none()prime(none()) = none()prime(Type1 ,Type2) = prime(Type1) | prime(Type2)

prime(Type1 | Type2) = prime(Type1) | prime(Type2)

prime(Type?) = prime(Type)prime(Type+) = prime(Type)prime(Type*) = prime(Type)

FLWOR Expressions:

Here is how to compute prime types:

quant(ItemType) = 1quant(empty()) = ?quant(none()) = 1quant(Type1 ,Type2) = quant(Type1) , quant(Type2)

quant(Type1 | Type2) = quant(Type1) | quant(Type2)

quant(Type?) = quant(Type) ?quant(Type+) = quant(Type) +quant(Type*) = quant(Type) *

FLWOR Expressions:

Here is how to compute quantifiers:

FLWOR Expressions:

Factorization tables:

,1?+*1++++?+*+*+++++*+*+*

|1?+*11?+*???**++*+******

1?+*11?+*???**++*+******

FLWOR Expressions:

Example:

prime(((xs:integer, xs:string) | xs:integer)* )= xs:integer | xs:stringprime(element(title), element(author)+ )= element(title) | element(author)

quant((xs:integer, xs:string) | xs:integer)*) = *quant(element(title) , element(author)+) = +

FLWOR Expressions:

Theorem – Factorization

For all type we have Type subtype prime(Type) quant(Type)

Further ifType subtype Prime Quantifier

thenprime(Type) subtype Prime and quant(Type) ≤ Quantifier

FLWOR Expressions:

statEnv |- Expr0 : Type

statEnv + varType(Var:prime(Type0)) |- Expr1 : Type1____

statEnv |- for $Var in Expr0 return Expr1 : Type1 quant(Type0)

Thank you:

THE END…