Queries in general-purpose programming languages

Languages

Theory Relational algebra Monads and comprehensions

Haskell and Links Monad comprehensions Comprehensions with group and order

C# and F# Language integrated query (LINQ) Extensible query syntax in F# 3.0

Problems

Expressing queries What operations can we use? What is the right syntax for queries? Should the syntax be extensible?

Compiling queries How to turn normal code to SQL? How to turn normal code to efficient SQL? Dealing with functions in SQL code

Theory

Relational algebra Selection () and projection () Cartesian product () Set union () and difference ()

Monad comprehensions Monadic bind [ … | x <- <e> ] Monadic return [ <e> | … ] Conditional [ … | …, <e> ]

Monads and monad comprehensions in Haskell

Monad comprehensions

Query example

Translation using monads bind return zero

[ p.Name | p <- db.Products, p.Price > 10 ]

db.Products `bind` (\p -> if (p.Price > 10) then return p.Name else zero)

Monadic bind and joins

Join using comprehensions

Translation using monads

Multiple uses of bind are nested Second data source may depend on values from first!

[ (p.Name, c.CategoryName) | p <- db.Products, c <- db.Categories , p.CategoryID = c.ID ]

db.Products `bind` (\p -> db.Categories `bind` (\c -> if (p.CategoryID = c.ID) then return (p.Name, c.CategoryName) else zero))

Comparing bind and join

Standard join Two independent data sources Equality condition on some projected values

But bind allows more Second data source may depend on first Can use arbitrary conditions

[ (p, c) | p <- db.Products, c <- db.Categories , p.CategoryID = c.ID ]

[ (p1.Name, p2.Name) | p1 <- Products , p2 <- [ p | p <- Products, p.Price > p1.Price ] ]

Adding ordering and grouping

“Ordering” signature Functions of type like:

reverse, orderBy (also needs a key selector)take, takeWhile, skip, … (also needs count or predicate)

“Grouping” signature Functions of type like:

groupBy, groupAdjacent (also needs a key selector)

Are we missing anything else?

Adding ordering and groupingOrdering syntax

The function to use follows then keyword by clause is optional argument (of type

Grouping syntax

Grouping function implicit, but can be set with using The by clause is optional

[ p | p <- db.Products, then reverse ][ p | p <- db.Products, then order by p.Price ]

[ p | …, then group by p.CategoryID ][ p | …, then group by p.CategoryID using groupAdj ]

SQL-style relationalqueries in LINQ using C#

LINQ Overview

Query example

Translating to SQL All operations have a corresponding method

Lambda function translated as expression tree Expression trees composed and processed

from p in db.Products join c in db.Categories on p.CategoryID equals c.IDselect new { p.Name, c.CategoryName }

db.Products.Where(p => p.Price > 10)db.Products.Select(p => p.Name)

Expressing queriesSelection and projection

Ordering

Grouping

Joins (products with selection)

from p in db.Products where p.Price > 10 select pfrom p in db.Products select p.Name

from p in db.Products orderby p.Price select p

from p in db.Products group p by p.CategoryID

from p in db.Products join c in db.Categories on p.CategoryID equals c.IDselect new { ... }

Customizing LINQ operators

Operations depend on the data source Syntax is translated to method calls

Wrap collection and then use different operator

Various applications LINQ over events, Parallel implementation

from p in db.Products group p by p.CategoryIDdb.Products.GroupBy(p => p.CategoryID)

from p in db.Products.WithAdjacentGrouping() group p by p.CategoryIDWithAdjacentGrouping : IE<T> -> IEWithGroup<T>GroupBy : IEWithGroup<T> -> IEWithGroup<IEWithGroup<T>>

Writing joins in LINQ

Syntax for joins Independent sources; equality condition only

Using multiple from clauses Later from can depend on previous variables We can write arbitrary conditions

from p in db.Products join c in db.Categories on p.CategoryID equals c.IDselect …

from p in db.Products from c in db.Categories.Where(c => p.CategoryID == c.ID)where c.Name != "Stuff"

Joins and theoryMultiple from clauses Uses an operation similar to Bind

Can be implemented using Bind

Syntax for joins Can be defined using SelectMany, but not vice versa

Resembles operation of “applicative functors”

SelectMany : M<S> -> (S -> M<C>) -> (S * C -> R) -> M<R>

Bind : M<S> -> (S -> M<R>) -> M<R>StrengthL : V * M<S> -> M<V * S>

Join : M<A> -> M<B> -> (A -> K) -> (B -> K) -> (A * B -> R) -> M<R>

More flexible queries in F# 3.0

Queries in F# 3.0

Design between Haskell and C# Can define custom operations

Here orderBy and takeWhile are not built-in

Two kinds of custom operations Preserve variable scope (orderBy, where, …) Require re-binding (groupBy)

query { for p in db.Products do orderBy p.Price takeWhile (p.Price > 100) }

What can be done in F# 3.0?Better queries for OData Open standard for REST based queries Supports only subset of query operations

Expanding dependent tables explicitly

Wrong! We did not say we will use Awards

Other interesting areas: NoSQL, etc.

query { for title in db.Movies do select title.Awards }

query { for title in db.Movies do expand title.Awards select title.Awards }

Some questions about queries that I’m interested in…

Possible questionsAdvanced type checking Translating custom functions to SQL Limiting operations and allowed patterns (OData)

Expressivity What are other join-like operations? Aren’t monads and bind too powerful? Are we missing other important operations? What else is there than just lists…

Classical problems Optimizing queries, efficient compilation

References

Haskell and comprehensions Comprehensive Comprehensions:

comprehensions with "Order by" and "Group by" Bringing Back Monad Comprehensions Monad Comprehensions:

A Versatile Representation for Queries

LINQ, C# and F# C# Language Specification msdn.microsoft.com/en-us/library/bb397676.aspx tomasp.net/blog/idioms-in-linq.aspx