Functional Database Strategies at Scala Bay

Functional Database Strategies

Scala Bay • March 20, 2017

HelloScala Bay

by Jason Swartz@swartzrock

Step One

Buy A Functional Database

Silly! Databases Aremutable, not

functional!

Well?Aren’t they?

1. Functional DB’s In Scalaa. Doobie

2. Functional DB’s In The Databasea. Immutable Rowsb. Window Functionsc. Events, Not State

Agenda

Remember That YourDatabase

is mutable.

Avoid SharingYour State

● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST● Execute state changes at THE EDGE

Safe Database Interactions

DoobieA Typelevel Project

Are these steps? Or a Monad?

val xa: Transactor[Task] = DriverManagerTransactor[Task]( "org.postgresql.Driver", "jdbc:postgresql://localhost/issues", "u", "p")

case class Issue(id: Int, issueId: Int, title: String, assignee: Int, status: String)

val issue: Issue = sql""" select id, issue_id, title, assignee, status from issues where issue_id = ${req.issue_id} order by id desc limit 1 """ .query[Issue] .unique .transact(xa) .unsafePerformSync

● ResultSet to Case Class Conversion● Async IO modeling with Task (scalaz version)● Think in SQL, not an ORM● Plain SQL with sql"" interpolation● SQL functions supported

Why Doobie?

Agenda

Let’s Talk AboutImmutable Tables

Create-Read-Update-Delete

GET /issuesGET /issues/{id}POST /issuesPUT /issues/{id}

Issue Endpoints

How Do These EventsAffect The Database?

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=103. PUT /issues/1 done=true

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

Not Bad.

Do You Know How We Got Here?

Why is ‘assignee’ NULL?

Mutable Table Rows Lose History

Immutable Table Rows Keep Their History

Let’s Try To Lock Down

Our State

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

Tables Are Mutable, But Table Rows Should Be

Immutable

In Other Words, Tables Should Be

Append-Only

How Do You Make An Append-Only

Table?

One: Don’t Let Your DB User Make

Changes

Grant select, insert on issues to my-db-user;

-- tested on Postgresql

Thank You! Goodbye!

Two: Pick The RightColumns

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false)

1. GET /issues/1

+------+-------------+------------+------------+----------+-------+| id | created | issue_id | title | assignee | done |+------+-------------+------------+------------+----------+-------+| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |+------+-------------+------------+------------+----------+-------+| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+

+------+-------------+------------+------------+----------+-------+| 4 | 09-18 18:24 | 1 | Config ELB | NULL | true |+------+-------------+------------+------------+----------+-------+

select * from issueswhere issue_id = :issue_idorder by id desc limit 1;

What Defines AnIssue,

The Version or the Entity?

ISSUES+------+-------------+------------+------------+----------+-------+| id | created | issue_id | title | assignee | done |+------+-------------+------------+------------+----------+-------+| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |+------+-------------+------------+------------+----------+-------+| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+

create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false )

create table notifications ( id serial, created timestamp default now(), issue_row_id int references issues )

NOTIFICATIONS+------+-------------+--------------+| id | created | issue_row_id | +------+-------------+--------------+| 1 | 09-18 19:13 | 2 |+------+-------------+--------------+

Issue Notifications AreImmutableIn The Database

That’s The Basics Of Immutability

In Table Rows

Agenda

SQL:2003 expands Groups into Windows

Works Great InPostgresql

● Aggregation functions, eg sum(), rank(), avg()● Window definitions with over()● Grouping with partition by, order

Window Functions

+------+------------+------------+----------+-------+| id | issue_id | title | assignee | done |+------+------------+------------+----------+-------+| 1 | 1 | Config ELB | NULL | false |+------+------------+------------+----------+-------+| 2 | 1 | Config ELB | 10 | false |+------+------------+------------+----------+-------+| 3 | 2 | Bug to fix | 11 | false |+------+------------+------------+----------+-------+| 4 | 2 | Bug to fix | 11 | true |+------+------------+------------+----------+-------+

with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day')select * from latest_issues where row_number = 1

+------+------------+------------+----------+-------+------------+| id | issue_id | title | assignee | done | row_number |+------+------------+------------+----------+-------+------------+| 1 | 1 | Config ELB | NULL | false | 2 |+------+------------+------------+----------+-------+------------+| 2 | 1 | Config ELB | 10 | false | 1 |+------+------------+------------+----------+-------+------------+| 3 | 2 | Bug to fix | 11 | false | 2 |+------+------------+------------+----------+-------+------------+| 4 | 2 | Bug to fix | 11 | true | 1 |+------+------------+------------+----------+-------+------------+

That WasWindow

Functions

Agenda

You Know How To Maintain State

Do We Still Need State?

Let’s Talk AboutEvent-Sourcing

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | 10 | true |+------+-------------+------------+----------+-------+

Events

States

Now We’re Storing Events,

Not States

create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text)

1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’

+----+-------------+----------+------------+---------+| id | created | issue_id | originator | payload |+----+-------------+----------+------------+---------+| 14 | 09-18 18:50 | 1 | 4a482... | <...> |+----+-------------+----------+------------+---------+

Create Events AndSimulate The State

1. Create-Issue

Issue(“Config ELB”, null, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue

Issue(“Config ELB”, 10, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue3. Complete-Issue

Issue(“Config ELB”, 10, true);

Real Events

Virtual States

So Why UseEvent-Sourcing?

1. High Write Performance2. Potential for Command/Query Separation3. Auditable4. Replayable5. Undo-able6. Monitorable

Reasons For Event-Sourcing

It’s Like Having Control Over The Versions Of

Your State Changes

It’s Like Having Control Over The Versions Of

Your Data

It’s Like GitFor Your Data

1. Frankly, It’s Weird2. Requires Events. No Events, No Event-Sourcing.3. As Of March 2017, It’s Still Non-Standard

Reasons Against Event-Sourcing

Wait! We’reScala

developers!

Who Cares About BeingNon-Standard?

That About Sums Up Event Sourcing

That About Sums Up Database

Interactions

Okay, Actually That’s The Entire Talk

Unless There’s More Time

Scala Bay • March 20, 2017

by Jason Swartz@swartzrock

Thank You For Attending

THIS SPACELEFT BLANK

Functional Database Strategies at Scala Bay

Technology

Transcript of Functional Database Strategies at Scala Bay

Functional Scala II (in practice)

Functional programming with Scala

Who's More Functional: Kotlin, Groovy, Scala, or Java?

Building a Functional Stream in Scala

Thinking functional-in-scala

IoT on DCOS - Scala By the Bay 2015

Introduction to Functional Programming in Scala

An Introduction to NLP4L (Scala by the Bay / Big Data Scala 2015)

Getting Functional with Scala

Functional Programming in Scala Summer 2008 · Functional programming • One of the most interesting aims of Scala is the attempt to unify object-oriented and functional programming

Intro to Functional Programming in Scala

Ankara Jug - Practical Functional Programming with Scala

Demystifying functional programming with Scala

Scala: Object-Oriented Meets Functional, by Iulian Dragos

The Functional Web Scala and Lift — Functional Recipes for ...steve.vinoski.net/pdf/IC-Scala_and_Lift.pdf · Scala and Lift ments and accept as return types. Scala has lexical closures,

Introduction to Functional Programming with Scala

Scala Bay Meetup - The state of Scala code style and quality

Teach Yourself some Functional Programming with Scala

Introduction to Functional Programming (SCALA) John Woodward.

Functional Programming and Concurrency Patterns in Scala