Post on 04-Jul-2020
Modern SQL:Evolution of a dinosaur
Markus Winand
Kraków, 9-11 May 2018
Still using Windows 3.1? So why stick with
SQL-92?
@ModernSQL - https://modern-sql.com/ @MarkusWinand
SQL:1999
WITH (Common Table Expressions)
Understand
this first
WITH (non-recursive) The ProblemNested queries are hard to read:
SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)
Then this...
WITH (non-recursive) The ProblemNested queries are hard to read:
SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)
Then this...
WITH (non-recursive) The ProblemNested queries are hard to read:
SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)
Finally the first line makes sense
WITH (non-recursive) The ProblemNested queries are hard to read:
SELECT…FROM(SELECT…FROMt1JOIN(SELECT…FROM…)aON(…))bJOIN(SELECT…FROM…)cON(…)
CTEs are statement-scoped views:
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)
Keyword
WITH (non-recursive) Since SQL:1999
CTEs are statement-scoped views:
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)
Name of CTE and (here optional) column names
WITH (non-recursive) Since SQL:1999
CTEs are statement-scoped views:
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)
Definition
WITH (non-recursive) Since SQL:1999
CTEs are statement-scoped views:
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)
Introduces another CTE
Don't repeat WITH
WITH (non-recursive) Since SQL:1999
CTEs are statement-scoped views:
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)
May refer toprevious CTEs
WITH (non-recursive) Since SQL:1999
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)AS(SELECT…FROM…)
SELECT…FROMbJOINcON(…)
Third CTE
WITH (non-recursive) Since SQL:1999
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)AS(SELECT…FROM…)
SELECT…FROMbJOINcON(…)
No comma!
WITH (non-recursive) Since SQL:1999
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)AS(SELECT…FROM…)
SELECT…FROMbJOINcON(…)
Main query
WITH (non-recursive) Since SQL:1999
CTEs are statement-scoped views:
WITHa(c1,c2,c3)AS(SELECTc1,c2,c3FROM…),
b(c4,…)AS(SELECTc4,…FROMt1JOINaON(…)),
c(…)AS(SELECT…FROM…)
SELECT…FROMbJOINcON(…)
Read top down
WITH (non-recursive) Since SQL:1999
‣ Literate SQL
Organize SQL code toimprove maintainability
‣ Assign column names
to tables produced by valuesor unnest.
‣ Overload tables (for testing)
with queries hide tablesof the same name.
Use-CasesWITH (non-recursive)
https://modern-sql.com/use-case/literate-sql
https://modern-sql.com/use-case/naming-unnamed-columns
https://modern-sql.com/use-case/unit-tests-on-transient-data
WITH are the "private methods" of SQL
WITH is a prefix to SELECT
WITH queries are only visible in the SELECT they precede
WITH in detail: https://modern-sql.com/feature/with
WITH (non-recursive) In a Nutshell
AvailabilityWITH (non-recursive)19
9920
0120
0320
0520
0720
0920
1120
1320
1520
17
5.1 10.2 MariaDB8.0 MySQL
8.4 PostgreSQL3.8.3[0] SQLite
7.0 DB2 LUW9iR2 Oracle
2005[1] SQL Server[0]Only for top-level SELECT statements[1]Only allowed at the very begin of a statement. E.g. WITH...INSERT...SELECT.
WITHRECURSIVE (Common Table Expressions)
(This page is intentionally left blank)
WITHRECURSIVE The Problem
CREATETABLEt(idNUMERICNOTNULL,parent_idNUMERIC,…PRIMARYKEY(id))
Coping with hierarchies in the Adjacency List Model[0]
WITHRECURSIVE The Problem
[0] Hierarchies implemented using a “parent id” — see “Joe Celko’s Trees and Hierarchies in SQL for Smarties”
SELECT*FROMtASd0LEFTJOINtASd1ON(d1.parent_id=d0.id)LEFTJOINtASd2ON(d2.parent_id=d1.id)
Coping with hierarchies in the Adjacency List Model[0]
WITHRECURSIVE The Problem
WHEREd0.id=?
[0] Hierarchies implemented using a “parent id” — see “Joe Celko’s Trees and Hierarchies in SQL for Smarties”
SELECT*FROMtASd0LEFTJOINtASd1ON(d1.parent_id=d0.id)LEFTJOINtASd2ON(d2.parent_id=d1.id)
Coping with hierarchies in the Adjacency List Model[0]
WITHRECURSIVE The Problem
WHEREd0.id=?
[0] Hierarchies implemented using a “parent id” — see “Joe Celko’s Trees and Hierarchies in SQL for Smarties”
SELECT*FROMtASd0LEFTJOINtASd1ON(d1.parent_id=d0.id)LEFTJOINtASd2ON(d2.parent_id=d1.id)
WITHRECURSIVE Since SQL:1999
WHEREd0.id=?
WITHRECURSIVEd(id,parent,…)AS(SELECTid,parent,…FROMtblWHEREid=?UNIONALLSELECTid,parent,…FROMdJOINtblON(tbl.parent=d.id))SELECT*FROMsubtree
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
Keyword
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
Column list mandatory here
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
Executed first
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
Result sent there
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
Result visible twice
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
n---123(3rows)
Once it becomes
part of the final result
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
n---123(3rows)
Second leg of UNION is executed
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
n---123(3rows)
It's a loop!
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
n---123(3rows)
It's a loop!
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
n---123(3rows)
It's a loop!
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
n---123(3rows)
n=3 doesn't match
Since SQL:1999WITHRECURSIVE
Recursive common table expressions may refer to themselves in a leg of a UNION[ALL]:
WITHRECURSIVEcte(n)AS(SELECT1UNIONALLSELECTn+1FROMcteWHEREn<3)SELECT*FROMcte
n---123(3rows)
n=3 doesn't matchLoop terminates
Since SQL:1999WITHRECURSIVE
AvailabilityWITHRECURSIVE19
9920
0120
0320
0520
0720
0920
1120
1320
1520
17
5.1 10.2 MariaDB8.0 MySQL
8.4 PostgreSQL3.8.3[0] SQLite
7.0 DB2 LUW11gR2 Oracle
2005 SQL Server[0]Only for top-level SELECT statements
SQL:2003
OVER and
PARTITIONBY
OVER (PARTITION BY) The ProblemTwo distinct concepts could not be used independently:
‣Merge rows with the same key properties
‣ GROUPBY to specify key properties
‣ DISTINCT to use full row as key
‣ Aggregate data from related rows ‣ Requires GROUPBY to segregate the rows
‣ COUNT, SUM, AVG, MIN, MAX to aggregate grouped rows
SELECTc1,SUM(c2)totFROMtGROUPBYc1
OVER (PARTITION BY) The Problem
Yes ⇠
Mer
ge ro
ws ⇢
No
No ⇠ Aggregate ⇢ Yes
SELECTc1,c2FROMt
SELECTDISTINCTc1,c2FROMt
SELECTc1,c2FROMt
SELECTc1,SUM(c2)totFROMtGROUPBYc1
SELECTc1,SUM(c2)totFROMtGROUPBYc1
OVER (PARTITION BY) The Problem
Yes ⇠
Mer
ge ro
ws ⇢
No
No ⇠ Aggregate ⇢ Yes
SELECTc1,c2FROMt
SELECTDISTINCTc1,c2FROMt
SELECTc1,c2FROMtJOIN()taON(t.c1=ta.c1)
SELECTc1,SUM(c2)totFROMtGROUPBYc1
,tot
SELECTc1,SUM(c2)totFROMtGROUPBYc1
OVER (PARTITION BY) The Problem
Yes ⇠
Mer
ge ro
ws ⇢
No
No ⇠ Aggregate ⇢ Yes
SELECTc1,c2FROMt
SELECTDISTINCTc1,c2FROMt
SELECTc1,c2FROMtJOIN()taON(t.c1=ta.c1)
SELECTc1,SUM(c2)totFROMtGROUPBYc1
,tot
SELECTc1,SUM(c2)totFROMtGROUPBYc1
OVER (PARTITION BY) Since SQL:2003
Yes ⇠
Mer
ge ro
ws ⇢
No
No ⇠ Aggregate ⇢ Yes
SELECTc1,c2FROMt
SELECTDISTINCTc1,c2FROMt
SELECTc1,c2FROMt
FROMt
,SUM(c2)OVER(PARTITIONBYc1)
SELECTdep,salary,SUM(salary)OVER()FROMemp
dep salary ts1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000
OVER (PARTITION BY) How it works
SELECTdep,salary,SUM(salary)OVER()FROMemp
dep salary ts1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000
OVER (PARTITION BY) How it works
SELECTdep,salary,SUM(salary)OVER()FROMemp
dep salary ts1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000
OVER (PARTITION BY) How it works
SELECTdep,salary,SUM(salary)OVER()FROMemp
dep salary ts1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000
OVER (PARTITION BY) How it works
SELECTdep,salary,SUM(salary)OVER()FROMemp
dep salary ts1 1000 600022 1000 600022 1000 6000333 1000 6000333 1000 6000333 1000 6000
OVER (PARTITION BY) How it works
SELECTdep,salary,SUM(salary)OVER()FROMemp
dep salary ts1 1000 100022 1000 200022 1000 2000333 1000 3000333 1000 3000333 1000 3000
OVER (PARTITION BY) How it works
)PARTITIONBYdep
OVER and
ORDERBY(Framing & Ranking)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
OVER (ORDER BY) The Problem
SELECTid,value,FROMtransactionst
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
OVER (ORDER BY) The Problem
SELECTid,value,
(SELECTSUM(value)FROMtransactionst2WHEREt2.id<=t.id)
FROMtransactionst
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
OVER (ORDER BY) The Problem
SELECTid,value,
(SELECTSUM(value)FROMtransactionst2WHEREt2.id<=t.id)
FROMtransactionst
Range segregation (<=)not possible with
GROUP BY orPARTITION BY
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYid
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDING
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +30
22 3 -10 +20
333 4 +50 +70
333 5 -30 +40
333 6 -20 +20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10
22 2 +20
22 3 -10
333 4 +50
333 5 -30
333 6 -20
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
OVER (ORDER BY) Since SQL:2003
SELECTid,value,
FROMtransactionst
SUM(value)OVER(
)
acnt id value balance
1 1 +10 +10
22 2 +20 +20
22 3 -10 +10
333 4 +50 +50
333 5 -30 +20
333 6 -20 .0
ORDERBYidROWSBETWEENUNBOUNDEDPRECEDINGANDCURRENTROW
PARTITIONBYacnt
OVER (ORDER BY) Since SQL:2003With OVER(ORDERBYn) a new type of functions make sense:
n ROW_NUMBER RANK DENSE_RANK PERCENT_RANK CUME_DIST1 1 1 1 0 0.252 2 2 2 0.33… 0.753 3 2 2 0.33… 0.754 4 4 3 1 1
‣ Aggregates without GROUPBY
‣ Running totals, moving averages
‣ Ranking‣ Top-N per Group
‣ Avoiding self-joins
[… many more …]
Use Cases
SELECT*FROM(SELECTROW_NUMBER()OVER(PARTITIONBY…ORDERBY…)rn,t.*FROMt)numbered_tWHERErn<=3
AVG(…)OVER(ORDERBY…ROWSBETWEEN3PRECEDINGAND3FOLLOWING)moving_avg
OVER (SQL:2003)
OVER may follow any aggregate function
OVER defines which rows are visible at each row
OVER() makes all rows visible at every row
OVER(PARTITIONBY …) segregates like GROUPBY
OVER(ORDERBY…BETWEEN) segregates using <, >
In a NutshellOVER (SQL:2003)
OVER (SQL:2003) Availability19
9920
0120
0320
0520
0720
0920
1120
1320
1520
17
5.1 10.2 MariaDB8.0 MySQL
8.4 PostgreSQLSQLite
7.0 DB2 LUW8i Oracle
2005 SQL Server
Hive
ImpalaSpark
NuoDB
SQL:2006
XMLTABLE
SELECTid,c1,nFROMtbl,XMLTABLE('/d/e'PASSINGxCOLUMNSidINTPATH'@id',c1VARCHAR(255)PATH'c1',nFORORDINALITY)r
XMLTABLE Since SQL:2006Stored in tbl.x:
<d><eid="42"><c1>…</c1></e></d>
XPath* expression to identify rows
*Standard SQL allows XQuery
SELECTid,c1,nFROMtbl,XMLTABLE('/d/e'PASSINGxCOLUMNSidINTPATH'@id',c1VARCHAR(255)PATH'c1',nFORORDINALITY)r
XMLTABLE Since SQL:2006Stored in tbl.x:
<d><eid="42"><c1>…</c1></e></d>
*Standard SQL allows XQuery
SELECTid,c1,nFROMtbl,XMLTABLE('/d/e'PASSINGxCOLUMNSidINTPATH'@id',c1VARCHAR(255)PATH'c1',nFORORDINALITY)r
XMLTABLE Since SQL:2006Stored in tbl.x:
<d><eid="42"><c1>…</c1></e></d>
*Standard SQL allows XQuery
XPath* expressions to extract data
SELECTid,c1,nFROMtbl,XMLTABLE('/d/e'PASSINGxCOLUMNSidINTPATH'@id',c1VARCHAR(255)PATH'c1',nFORORDINALITY)r
XMLTABLE Since SQL:2006Stored in tbl.x:
<d><eid="42"><c1>…</c1></e></d>
*Standard SQL allows XQuery
Row number (like for unnest)
SELECTid,c1,nFROMtbl,XMLTABLE('/d/e'PASSINGxCOLUMNSidINTPATH'@id',c1VARCHAR(255)PATH'c1',nFORORDINALITY)r
XMLTABLE Since SQL:2006Stored in tbl.x:
<d><eid="42"><c1>…</c1></e></d>
*Standard SQL allows XQuery
Result id|c1|n----+----+---42|…|1
XMLTABLE Availability19
99
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDBMySQL
10[0] PostgreSQLSQLite
9.7 DB2 LUW11gR1 Oracle
SQL Server[0]No XQuery (only XPath). No default namespace declaration.
SQL:2008
FETCHFIRST
SELECT*FROM(SELECT*,ROW_NUMBER()OVER(ORDERBYx)rnFROMdata)numbered_dataWHERErn<=10
FETCHFIRST The ProblemLimit the result to a number of rows. (LIMIT, TOP and ROWNUM are all proprietary)
SQL:2003 introduced ROW_NUMBER() to number rows.But this still requires wrapping to limit the result.
And how about databases not supporting ROW_NUMBER()?
SELECT*FROM(SELECT*,ROW_NUMBER()OVER(ORDERBYx)rnFROMdata)numbered_dataWHERErn<=10
FETCHFIRST The ProblemLimit the result to a number of rows. (LIMIT, TOP and ROWNUM are all proprietary)
SQL:2003 introduced ROW_NUMBER() to number rows.But this still requires wrapping to limit the result.
And how about databases not supporting ROW_NUMBER()?
Dammit! Let's takeLIMIT
SELECT*FROMdataORDERBYxFETCHFIRST10ROWSONLY
FETCHFIRST Since SQL:2008SQL:2008 introduced the FETCHFIRST…ROWSONLY clause:
FETCHFIRST Availability19
99
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 MariaDB3.19.3[0] MySQL
6.5[1] 8.4 PostgreSQL2.1.0[1] SQLite
7.0 DB2 LUW12cR1 Oracle
7.0[2] 2012 SQL Server[0]Earliest mention of LIMIT. Probably inherited from mSQL[1]Functionality available using LIMIT[2]SELECTTOPn... SQL Server 2000 also supports expressions and bind parameters
SQL:2011
OFFSET
SELECT*FROM(SELECT*,ROW_NUMBER()OVER(ORDERBYx)rnFROMdata)numbered_dataWHERErn>10andrn<=20
OFFSET The ProblemHow to fetch the rows after a limit?
(pagination anybody?)
SELECT*FROMdataORDERBYxOFFSET10ROWSFETCHNEXT10ROWSONLY
OFFSET Since SQL:2011SQL:2011 introduced OFFSET, unfortunately!
SELECT*FROMdataORDERBYxOFFSET10ROWSFETCHNEXT10ROWSONLY
OFFSET Since SQL:2011SQL:2011 introduced OFFSET, unfortunately!
OFFSETGrab coasters & stickers!
https://use-the-index-luke.com/no-offset
OFFSET Since SQL:201119
99
2001
2003
2005
2007
2009
2011
2013
2015
5.1 MariaDB3.20.3[0] 4.0.6[1] MySQL
6.5 PostgreSQL2.1.0 SQLite
9.7[2] 11.1 DB2 LUW12c Oracle
2012 SQL Server[0]LIMIT[offset,]limit: "With this it's easy to do a poor man's next page/previous page WWW application."[1]The release notes say "Added PostgreSQL compatible LIMIT syntax"[2]Requires enabling the MySQL compatibility vector: db2setDB2_COMPATIBILITY_VECTOR=MYS
OVER
WITHnumbered_tAS(SELECT*)
SELECTcurr.*,curr.balance-COALESCE(prev.balance,0)FROMnumbered_tcurrLEFTJOINnumbered_tprevON(curr.rn=prev.rn+1)
OVER (SQL:2011) The ProblemDirect access of other rows of the same window is not possible.
(E.g., calculate the difference to the previous rows)
currbalance … rn
50 … 190 … 270 … 330 … 4
FROMt
WITHnumbered_tAS(SELECT*)
SELECTcurr.*,curr.balance-COALESCE(prev.balance,0)FROMnumbered_tcurrLEFTJOINnumbered_tprevON(curr.rn=prev.rn+1)
OVER (SQL:2011) The ProblemDirect access of other rows of the same window is not possible.
(E.g., calculate the difference to the previous rows)
currbalance … rn
50 … 190 … 270 … 330 … 4
FROMt,ROW_NUMBER()OVER(ORDERBYx)rn
WITHnumbered_tAS(SELECT*)
SELECTcurr.*,curr.balance-COALESCE(prev.balance,0)FROMnumbered_tcurrLEFTJOINnumbered_tprevON(curr.rn=prev.rn+1)
OVER (SQL:2011) The ProblemDirect access of other rows of the same window is not possible.
(E.g., calculate the difference to the previous rows)
currbalance … rn
50 … 190 … 270 … 330 … 4
FROMt,ROW_NUMBER()OVER(ORDERBYx)rn
WITHnumbered_tAS(SELECT*)
SELECTcurr.*,curr.balance-COALESCE(prev.balance,0)FROMnumbered_tcurrLEFTJOINnumbered_tprevON(curr.rn=prev.rn+1)
OVER (SQL:2011) The ProblemDirect access of other rows of the same window is not possible.
(E.g., calculate the difference to the previous rows)
currbalance … rn
50 … 190 … 270 … 330 … 4
FROMt,ROW_NUMBER()OVER(ORDERBYx)rn
prevbalance … rn
50 … 190 … 270 … 330 … 4
WITHnumbered_tAS(SELECT*)
SELECTcurr.*,curr.balance-COALESCE(prev.balance,0)FROMnumbered_tcurrLEFTJOINnumbered_tprevON(curr.rn=prev.rn+1)
OVER (SQL:2011) The ProblemDirect access of other rows of the same window is not possible.
(E.g., calculate the difference to the previous rows)
currbalance … rn
50 … 190 … 270 … 330 … 4
FROMt,ROW_NUMBER()OVER(ORDERBYx)rn
prevbalance … rn
50 … 190 … 270 … 330 … 4
WITHnumbered_tAS(SELECT*)
SELECTcurr.*,curr.balance-COALESCE(prev.balance,0)FROMnumbered_tcurrLEFTJOINnumbered_tprevON(curr.rn=prev.rn+1)
OVER (SQL:2011) The ProblemDirect access of other rows of the same window is not possible.
(E.g., calculate the difference to the previous rows)
currbalance … rn
50 … 190 … 270 … 330 … 4
FROMt,ROW_NUMBER()OVER(ORDERBYx)rn
prevbalance … rn
50 … 190 … 270 … 330 … 4
+50+40-20-40
SELECT*,balance-COALESCE(LAG(balance)OVER(ORDERBYx),0)FROMt
Available functions:LEAD/LAGFIRST_VALUE/LAST_VALUENTH_VALUE(col,n)FROMFIRST/LASTRESPECT/IGNORENULLS
OVER (SQL:2011) Since SQL:2011SQL:2011 introduced LEAD, LAG, NTH_VALUE, … for that:
OVER (LEAD, LAG, …) Since SQL:201119
9920
0120
0320
0520
0720
0920
1120
1320
1520
17
5.1 10.2[0] MariaDB8.0[0] MySQL
8.4[0] PostgreSQLSQLite
9.5[1] 11.1 DB2 LUW8i[1] 11gR2 Oracle
2012[1] SQL Server[0]No IGNORENULLS and FROMLAST[1]No NTH_VALUE
System Versioning (Time Traveling)
INSERTUPDATEDELETE
are DESTRUCTIVE
System Versioning The Problem
CREATETABLEt(...,start_tsTIMESTAMP(9)GENERATEDALWAYSASROWSTART,end_tsTIMESTAMP(9)GENERATEDALWAYSASROWEND,
PERIODFORSYSTEM_TIME(start_ts,end_ts))WITHSYSTEMVERSIONING
System Versioning Since SQL:2011Table can be system versioned, application versioned or both.
ID Data start_ts end_ts1 X 10:00:00
UPDATE...SETDATA='Y'...
ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00
DELETE...WHEREID=1
INSERT...(ID,DATA)VALUES(1,'X')
System Versioning Since SQL:2011
ID Data start_ts end_ts1 X 10:00:00
UPDATE...SETDATA='Y'...
ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00
DELETE...WHEREID=1
ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00 12:00:00
INSERT...(ID,DATA)VALUES(1,'X')
System Versioning Since SQL:2011
Although multiple versions exist, only the “current” one is visible per default.
After 12:00:00, SELECT*FROMt doesn’t return anything anymore.
ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00 12:00:00
System Versioning Since SQL:2011
ID Data start_ts end_ts1 X 10:00:00 11:00:001 Y 11:00:00 12:00:00
With FOR…ASOF you can query anything you like: SELECT*FROMtFORSYSTEM_TIMEASOFTIMESTAMP'2015-04-0210:30:00'
ID Data start_ts end_ts
1 X 10:00:00 11:00:00
System Versioning Since SQL:2011
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 MariaDB[0]
MySQLPostgreSQLSQLite
10.1[1] DB2 LUW10gR1[2] Oracle
2016 SQL Server[0]Available in MariaDB 10.3 beta.[1]Third column required (tx id), history table required.[2]Functionality available using Flashback
System Versioning Since SQL:2011
SQL:2016 (released: 2016-12-15)
LISTAGG
Since SQL:2016
grp val1 B1 A1 C2 X
grp val1 A, B, C2 X
SELECTgrp,LISTAGG(val,',')WITHINGROUP(ORDERBYval)FROMtGROUPBYgrp
LISTAGG(val,','ONOVERFLOWTRUNCATE'...'WITHCOUNT)➔'A,B,...(1)'
LISTAGG(val,','ONOVERFLOWERROR)
Default
LISTAGG
LISTAGG(val,','ONOVERFLOWTRUNCATE'...'WITHOUTCOUNT)➔'A,B,...'
Default
1999
2001
2003
2005
2007
2009
2011
2013
2015
5.1[0] MariaDB4.1[0] MySQL
7.4[1] 8.4[2]9.0[3] PostgreSQL3.5.4[4] SQLite
10.5[5] DB2 LUW11gR1 12cR2 Oracle
SQL Server[6]
[0]group_concat[1]array_to_string[2]array_agg[3]string_agg[4]group_concat without ORDER BY[5]No ON OVERFLOW clause[6]string_agg announced for vNext
LISTAGG Availability
[0] group_concat [1] array_to_string [2] array_agg [3] string_agg
[4] group_concat w/o ORDERBY [5] No ONOVERFLOW clause [6] string_agg announced for vNext
New in SQL:2016 JSON
LISTAGGhttps://modern-sql.com/feature/listagg
ROW PATTERN MATCHING https://www.slideshare.net/MarkusWinand/row-pattern-matching-in-sql2016
DATE FORMAT POLYMORPHIC TABLE FUNCTIONS
➔ https://modern-sql.com/blog/2017-06/whats-new-in-sql-2016
SQL has evolved beyond the relational idea.
Modern SQL? @MarkusWinand
SQL has evolved beyond the relational idea.
If you are using SQL like 25 years ago,you are doing it wrong!
Modern SQL? @MarkusWinand
SQL has evolved beyond the relational idea.
If you are using SQL like 25 years ago,you are doing it wrong!
A lot has happened since SQL-92.
Modern SQL? @MarkusWinand
https://www.flickr.com/photos/mfoubister/25367243054/
I have shown you a few features today
https://www.flickr.com/photos/mfoubister/25367243054/
I have shown you a few features today
https://www.flickr.com/photos/mfoubister/25367243054/
There are hundreds more to discover
@ModernSQL modern-sql.comMy other website:
https://use-the-index-luke.com
Training & co: https://winand.at/