Composable Queries with Ecto Drew Olson

@drewolson

* Brief Ecto Intro * Query Expressions * Composition * Query Pipelines

Please ask questions

Intro

Patterns from real applications

A few models for examples

Intro - Models

defmodule MyApp.Post do  use Ecto.Model  import Ecto.Query

  schema "posts" do    field :body, :string    field :published, :boolean    field :published_at, Ecto.Date    field :title, :string

    has_many :comments, MyApp.Comment  endend

Intro - Models

defmodule MyApp.Comment do  use Ecto.Model  import Ecto.Query

  schema "comments" do field :body, :string    field :commenter, :string    field :votes, :integer

    belongs_to :post, MyApp.Post  endend

Keyword query syntax

Intro - Baby’s First Query

posts = MyApp.Repo.all(  from p in MyApp.Post,  where: p.published == true)

This is how I started writing Ecto queries

Intro

Separate construction and execution

Intro

Intro - First Query Deconstructed

query = from p in MyApp.Post,        where: p.published == true

MyApp.Repo.all(query)

Intro - Fancy Query

query = from c in MyApp.Comment,        join: p in assoc(c, :post),        where: p.id == 1 and c.votes > 5        select: c

comments = MyApp.Repo.all(query)

Query Expressions

Query Expressions - Translation

query = from p in MyApp.Post,        where: p.published == true

query = where(MyApp.Post, [p], p.published == true)

Query Expressions - Fancy

query = from c in MyApp.Comment,        join: p in assoc(c, :post),        where: p.id == 1 and c.votes > 5        select: c

query = MyApp.Comment|> join(:left, [c], p in assoc(c, :post))|> where([_, p], p.id == 1)|> where([c, _], c.votes > 5)|> select([c, _], c)

Composition

Both query styles are composable.

Queries can be the “subject” of new queries.

Composition

Composition

query1 = MyApp.Comment

query2 = from c in query1,         where: c.votes > 5

query3 = from c in query2,         join: p in assoc(c, :post),         where: p.id == 1,         select: c

MyApp.Repo.all(query3)

Composition

query1 = MyApp.Comment

query2 = query1|> where([c], c.votes > 5)

query3 = query2|> join(:left, [c], p in assoc(c, :post))|> where([_, p], p.id == 1)|> select([c, _], c)

MyApp.Repo.all(query3)

We can now extract reusable components, name them and compose them.

Composition

Compositiondefmodule MyApp.Comment do  ...

  def for_post(query, id) do    from c in query,    join: p in assoc(c, :post),    where: p.id == ^id,    select: c  end

  def popular(query) do    from c in query,    where: c.votes > 5  endend

alias MyApp.Comment

Comment|> Comment.popular|> Comment.for_post(1)|> MyApp.Repo.all

Composition - Reads so nice :)

Comment|> Comment.popular|> Comment.for_post(1)|> MyApp.Repo.all

Query Pipelines

Query Pipelines

Comment|> Comment.popular|> Comment.for_post(1)|> MyApp.Repo.all

Query Pipelines

Comment|> Comment.popular|> Comment.for_post(1)|> MyApp.Repo.all

<- source<- transformation

<- transformation<- sink

Query Pipelines - A note

Welcome to my typespecs, where the types are all made up and the points don’t matter.

Query Pipelines - Source

A source is the starting point for a query.

@spec source() :: Query.t

Query Pipelines - Source Examples

MyApp.Post

MyApp.Post.owned_by(user)

Query Pipelines - Transformation

A transformation expands or constrains an existing query.

@spec transformation(Query.t) :: Query.t

Query Pipelines - Transformation Examples

MyApp.Post.published(query)

MyApp.Comment.for_post(query, post)

Query Pipelines - Sink

A sink executes a query and returns a result.

@spec sink(Query.t) :: Result.t

Query Pipelines - Sink Examples

MyApp.Repo.all(query)

MyApp.Repo.one(query)

MyApp.Repo.paginate(query)

Query Pipelines

Comment|> Comment.popular|> Comment.for_post(1)|> MyApp.Repo.all

Query Pipelines

Pipelines are fractal

Query Pipelines - Fractal

MyApp.Repo.paginate(query)

Query Pipelines

Pagination acts as a sink, but is really several smaller pipelines

defmodule MyApp.Repo do  def paginate(query, page_number \\ 1) do    {entries(query, page_number), total_entries(query)}  end

  defp entries(query, page_number) do    page_size = 10    offset = page_size * (page_number - 1)

    query    |> limit([_], ^page_size)    |> offset([_], ^offset)    |> all  end

  defp total_entries(query) do    query    |> exclude(:order_by)    |> exclude(:preload)    |> exclude(:select)    |> select([_], count("*"))    |> one  endend

See Also

* blog.drewolson.org * hex.pm/packages/scrivener

Fin

Thanks. Questions?

@drewolson